DQCOHEMT BESUflE 



ED 112 871 

AOTHOE: 

IKSTITOTION 

SPONS AGENCY 

BEPOET NO ' ' • 
PUB DATE 
NOTE 

, EDBS PEICE 
-D^SCEIPTOBS 



IB 002 58-1 

°«' 

Mann; HiHiam C. ; And Others . 
Observation^ethods for Hunan Dialogue,- 
University of Southern ^California, Marina del Bey. 
Infornation Sciences Inst. 

Advanced Besearch Pro jects Agency (DOD), Washington, 
D • C •> 

ISI-EB-7Sr33 
30 Jun 75 
92p. 

MFrr$0.76 jAC'$H.H3 Plus postage 

Artificic(l/Tntelligence; **Communication (Thought 
Transfer) ; c<yiiSundcations; 'Computational Linguistics; 
♦Computer Science; Data Collection; Human . 
Development; Humanization ; Information The'ory; ♦Man 
Machine Systems; .♦Observation; Research • 
Methodology ' ^ 

Human Dialogue; -Natural Language •'■"V ' .1 



IDENTIFIEES 

ABSTEACT , " ' • . ■ . , 

„ . Progress is described ow a new approach to improve 

man-machine communication in order "to significantly expand and 
diversify the capabilities of the computer interfaces .that people 
use. The objective is to design computer processes that can 
assimilate particular aspects of dialogue between people, then- 
transfer the processes into man-iachine communication. Nine^ aspects 
of human ability to communicate are /iselected and studied in -detair- 
acc^rding tq „new methods of * data collection described in the report. 
Explicit observationfeil instruction^ .also are given. (Author/SK) 



7 




♦ ; Doculaents acquired by|EEIC include 1»any inforfel unpublished 

♦ materials not available f<tom other sources. -EEIC makes every^fort 

♦ to obtain the best copy available. Nevertheless, items of ^marginal 

♦ reproducibility are bfte|i encountered a.nd this affects the quality 

♦ of the microfiche^and hafcdcopy reproductions .EEIC makes available 

♦ Via the EEIC Document Eeproduction Service (EDES). EDES is not 
responsible for the quality' of the original document. EeprodUctions ♦/ 
supplifed by EDES are thfe best that /can be made from the original. ♦ 




\ 



Wjlliam C. Mann 
James A. /Moore - 
James A.l,evin 
James H. Carlisle ; 




5? 

O 

f\ UNIVERSITY OF SOUtHEltN CALIFORNIA \ 



^ ARP A ORDER NO. 2m 



^ ; ISl/RR-75'33 
^ fmie m 



Observation Methods for Human Dialbgue 



- . i 



/ 



//■„ 



r — 



II)f^QRMAffON ^ENCPS INSTlTUTri 



y • II , . . • — ■ 

4676 Admmtlty Way/ Marma del ^ 

(2J3) 822-hll ' 



P>-^/w>-^//«« ^^^^^^ was smarted by the Office of Naval Research, Personnel and TroiniL Research Programs; CodM^, under 
Contract N00014-75-C-Q710, Nkl54-374rmid^ f ' r 

r* ' Jf^'^y^'^/. and_concliisions contained this docutfient are those of the.author(s) and should tpt-be interpreted as necessarily Apre- 
Wrn i/--'*^ '^^W^flPof'""' ^"Pf-^P^^ or implied, of the Office of Nival Research,- the Defense Advanced Research projects 
9 \2YA{^»^y> or the US. Government. \ i ' /-M 

j^^^' doctiment is approved for public relme and sale; distribution is unlimited. , ' 



UNCLASSIFIED 



/ 



SECURITY CLASSIFICATION OF THIS PAcjlS fiyh> n Daf» g7r«f#<0 / 



REPORT DOCUMENtATION PAGE 



^ READ INSTRUCTIONS 
BEFORE COMPLETING FORM? 



I 1. REPORT NUMBER ^ 

ISI/RR-75-33 



1 4. TITLED (MneS SubUtl^) 



2. ^OVT ACCESSION NO 



HECIP^IENrs CATALOG NUMIER 



OBSERVATION METHODS FO^ HUMAN DIALOGUE 



5. TYfE or REPORT A PERIOD COVERED 

Technical - January 1, 1975 
through JuW 30, 1975 



«. ^ewrORMlNOORa. REPORT NUMiER 



Vnilam C. Mann, James A. Moore, James ^"A. Levin 
James H. Carlisle ^ 



t. CONTRACT OR GRANT NUHtERf*; 

NOOOIV75-C-0710 • 



PERFORMING ORGANIZATION NAME AND ADDRESS 

use/ Informal}! on Scjences Institute 

A676 Admrralty Way , 

Marina del Rey, California 9Q291 



10. PROGRAM ja.EMENT, PROJECT. TASK 
AREA * WORK UNIT NUMtERS 



RR042-06 



RR042-06-0I 
'NKI54-374 



Human7Resou?ces Research Office 
Advanced Research Projects Agency 
Hop Wilson BlvdJ, Arlington, W 22209 



12. REPORT DATE 

June 30, 1975 



13- NUMBER OF PAGES 

90 



1 AGENCY^NAME^a ADDRESSf// ditUrtit tram ControtUng Oiitc») 

Personnel and Training Research Programs 
^ Office of Naval Research - Code A58 ' 
800 N. Quincy Street, ' I 

Arlington, VA 22217 / 



15. SECURITY CLASS, (qi vport) 

Unclassified ' 



f|,f^,Jg£'^'<?AT|ON/'DOWNGRADINq 



16* DISTRIBUTION'STATEMENT (ojflthijt Report) 



Vp roved for publj|- release; distribution uriTImltei. 




DISTRIBUTION STATEMENT Y©/ (A* Mbatrmct miffditi Stock 20, It dUUrmt ttom Report) 

• t7 



U. SUPPLEMENTARY NOTE? 



KEY WORDS rConfinu* .on r*v>r** s/^* // n,«e*#«ary md Identity by btocic number) [ 

dialogue, 'man-machine ^^^^ computer, pfocess ^lodel, natural. 

aHguage, data collection, human communlcation/artificial intelligence 
IlJhoic*'"; c°niP"ter science, group behavior; coding 

methods, reference, topic, , quest ion , request^ compreh^ens'fon, understand ing,, 
correction .g^paraphrase. ■ II 



^BSTRACT (Contirfu^ on r#v#r#i aldm It n^c««««rx mtd ld»ntity by btoek ntmbT) 




DD 



1 JAN 73 



EDIT40N or KnOV 65 IS OBSOUETE 
S/Nf OW)2.014- 660I 



uncl)^ssified 



3 



SECURITY CLASSIFICATION OF THIS PAGE (Wh^n Dmtm Enft0d) 



UNCLASSIFIED 



SeCUWITY CUA»8tr)CfcTI0N or THU ^AGEf1Wi»n P«f KnftttQ 



20. ABSTRA.CT 



This report dp$crib^ progrps on a new approach, for improving man-machine 
communication. The goal oY the woVk is to significantly expand and diversify the capabilities of 
the computer interfaces that people use. .The approach is fir^t to design- computer processes 
that can assimilate particular aspects of dialogue between people, then to transfer these 
processes ihto man-machine communication. g / , ) . • 

The approach requires that particular aspects' of the human ability to cominunicate be^ 
selected and studied in detail. This report describes new methods of data collection developed 
to 'meet th^s need and tells how they will be used. - ° ■ "S. 

The report focuses on nine phenomena of human dialogue which hav^been 'selected fro 
. appt'oxirt^ately 23 phenomena proposed /nd explored • For. most of. the ' 
observational instructions are given as well. 

■ ' ' ' . V ■ 



nine, explicit 



J. 



<3l 



\ 



T 



UNCLASSIFIED 



ERIC 



SECURif y. CLASSIFlCAT10|i OF VhIS PAGEflfhffn 0«(« Enikfd) 



OBSERVATION METHODS FOR HUMAN^mALOdVE 



i 



n 



by 

» ^ William Mam 
^ James A. Moore 
• James A- Levin 
James H. Carlisle 

■ ■ , - 



June 1975 



U.S, DEPARTMENT OB HEALTH, 
EDUCATION &WELFAIIE 
. NATIONAL INSTITUTE OF 
EDUCATION 

THIS DOCUMENT HAS BEEN REPRCT 
DUCEO EXACTLY Af REt^lVEO FROM 
THE PERSON OR ORGANIZATION ORIGIN 
ATING IT POINTS OF VIEW OR OPINIONS 
S,TATEO 00 NOT NECESSARILY, REPRE 
SENT OFFICIAL NATIONAL INSTITUTE OF 
EDUCATION POSITION OR POLICY 



use/Information Sciences ln$titute 
4676 Admiralty Way ' • 
Marina Del Rey, California 90291 



The research reported herein i« supported, by the Personnel and Training Research 
• Programs of. the Office of Naval- Research (Code 458), Arlington VA 22217, Contract 
Authority Identification Number NR 154-374,, Contract NOOO 14-75-0-0710, under ARPA 
Order Number 2930 from the Hun^an 'Resoijrces Research Office of the Advanced 
Research Projects Agency. The r^'pbrt is a))"proved for public release; distribution 
unlimited. Reproduction in whole ortin part is-;permitte|d for any purpose of the United 
States Government. Opinions expre|Sed herein "are those of the ^authors and do not 
necessarily reflect policy of any agency of the United- States Government. 



coivrBJvrs 



Abstract • v * ^' 

k 

1 Overview of the Report 1 

2 The Research Environment of the Observer's Methods 

3 Understanding The Observer's Task * 

4 Phenomena for Observation 15 . 



I Observer I mtmction Sections 

*\ ■ . » . • • . ' ' 

5 Orientation for the Observer 19 . 

6 Repeated Reference and Text Reference 22 * 

7 Ffequests 31 ' ^ * ||j 

8 Expression of Comprehension 42 a, . 

9 Similar E)(pressions - two kinds 

\0 Tbpjt Structure 57 ' ^ ^ 

11 Incremental and Prerequisite Information * 

t ■ t. 

2 Correction Actions 65 * 



13 Summary and Plans 74 



APPENDIX: Full Text of the Dialogue Example 76 



References 85 . 

. .V- ■ 

■ ^ i 6 



, - ABSTRACT 



This report describes progress ot»- a new apfproach for improving man-machine 
communication. The goal of the work is to significantly expand and diversify the 
capabilities of the computer interfaces that people use. The approach is first to design 
^ computer, processe^that can a5.nmi/ale particular'a&pects..of dtalogue bptween people, 
then to transfer these processes into man-machine communication. 

^ The approach requires that particular asi)ec1s of the human ability jcj communicate 
be selected and studied in detail. This repl)rt describes new methods of data collection 
developed fo m*t this need anJi tells how they will be used. • 

The report focLjses on nine phenomena of hwman dialogue which have been selected 
from approximately 23 phenomena proposed and explored. For mpst of the ninb, 
explicit observational instructions are given as well. / 1 



/. OVKHVIEIV OF THE REPORT 



) 



This report is a progress report rather than a report of completed results/ While ' 
enough has been accomplished to be interesting, it is all'tentative and subject to^ 
change; therefore we simply exhibit it. ■ " " ^ ^ 

■ " ' , . o ■ ■ ' , ■ ■ ■ , , ' ' • . .- ■■ 

• '• - ' . ♦ . . • - 

Most of the report is a description of methods which a person, called an Observer, 
C9n use to identify and describe nine'kinds of phenomena which arise in human dialogue. 
The phenomena are: * ' - ; • 



Repeated Reference 
Requests 

» Expression of Comprehension 
Similar Expressions ^ two kinds 
Topic Strqcture J ^ 
IncremerttaV^nd Prereq 
Correction Actions . .. 5 



ll:: V 



The methods, are u^ed in building computer programs- ttiat xan interpret dialoiu^. We 

■ ' ■ , . • • ^' ■•• ' " ■■ ' . - ' 

expect that the subprdcesses of these programs will be able to improve man-machine 

■ ■ * ' ' - " . • . ■ ' i, ■ ■ "•..«'" 

communicatibn when they arfe JmplantiscI in ta§k^ori§ntad systems: 

More context* than we can supiply here is nee<fed fOj an accurate interpretation df 
the significance ^f the observation methods. The context-setting sections do not tell' 

the whole story, but we expect that most headers only want* a brief overview. The 

• ■ ■ ,-•<>.' 

, report does not contain a detailed presentation of the r^latfonship of this work to 
preceding and concurrent research. This has been done eteWh^re (Mann 1974 > and 
will be updated in forthcoming publications and reports. 
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Tha work reported here is not embedded ' in a traditional, we'l-worn approach to 
commilnication. . It is' a new' approach to the study of person-to-person communication, 
with particular attention to improving irJan-machine communication as a icorise4uencif 
The report deals almost exc^lusively with data-definition and datarcollection portijjns of 
the methodology, which are but a small part of the whole. The approach dr^w^ heavily 
on cbmputer science, linguistics; and psychology, and on Other discipiin^ to a 'lesser 
extent. .. / 



In. yet another way this report is not representative the whole. - It suggests a 
particular broad scope, of attention for which wje are now prepared to <Jo certain kirtds 

of data development. But we know thafihe whole scope cannot be approached at 

■. .. I ^ ■ . ■. ' ■ '■ - . 

, once; it must be done gradually, and the selectivity of that movement is not at all 

apparent in the report. \ 

. Finally, some cautions are in order about the Observation Instructions in the report. 
They are not really ready for direct use by anyone jpther than the authors. In 
developing them, primary attention was paW to what kinds of things to annotate rather, 
than to the notational forms 'or the presentation of ideas to the Observer. The major 
reason for this is that we expect the substance Of the instructions to change" 
significantly under the pressure of the next few months • of use. Any significant 
cosmetic or presentational cleanup at present would be premature, as woul^°any 
attempt to determine their reliability. 
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As. a progress report, it is^contfibution to empirical research on hui||an symbblic 
communication, and thereby tp man-machine communication 

During) the past two years, there has been a very substantial increase in 'the 
number oMn-line, interactive users of computers in the military. More important, many 
of the long-term military aspirations fpr command-control and for administrative .and 
support functions call for computer-based, on-line, interactive :systems. Thus this 
research es meeting a growing need.for effective, user-oriented, man-machine interfaces 
directly and effectively supporting military requirements. 



2. Tim^RBSEARCU ENVIRONMENT 
bf. ttiE OBSERVER'S METHODS 



. In this section we will relate the Observer's t^sk to our general mfetho^fogy. Each 
Observer <5/orkinr with a single dialogue) carries out one instance of the basic 
• experiment. Which in turn Will lea^ to a' .v,ndel for that particular dialogue. This bisic 
experiment is, to be repeated, many times with different ObseF^rs Tnd tdialogues. 
fHibse dialogue-comprehensioji processes which have repeatedly shown themselves 
valuable throughout these experiments will then constitute the overall results; they' ar« 
^ the primary candidates for the final step of embedding in t,asK-ori^nted . system 



applications. - \ . 

... i . • .■ r - ' ' ' 

. » • ' ■ * 

" THE BASIC EXPERIMENT . ' 

The basic experiment consists of four steps, as illustrated in Figure 2-1: 1) acquire 
transcript of dfalogue for study, 2) gather Observer's ^commentaries, 3^ construct a 
•; process model to atcount for these observations, and 4 compare the actions of the- 
processes in the model with the Observer's judgments and Observer's behaviors ^ 

Dialogue Tnanscripts ' ' ' '■ °~ 

We have chosen to deal with dialogues only;, we ' do not plan to cover cases .of 
, multi-person conversation.. The dialogue" must be in a maci^ne-readable representation. 
We will use;a con^ntipnally .typed transcript' of the te'xt of the dialogue, ^^ps with 



TRANSCRIP/T 



SI : Help me! 
S2: How?' 

ST : Whalf db .1 .. 



'Dialogpe 
comprehension 
procfess set 




S^eqi/errce of 



I memdry. states 

; ^.ofgT> S2 ^ 



Compare arid 
verify 



Figure 2-1' - Basic Dialogue Case, Experiment 
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' s(«ne.. ti,mrng infoKwation, We do not n6w envision any attempt to include'and use such 
subtleties.. as facral expression, gestures/and intonation." ^n bjerlo assure ourselves 
Wt the transcripts gre not systematically excluding aspects which were sjgnificCit 'in 
the "original cdnversatibn, we will examine only those dialogues which Wefe^-originally 
.conducted oyer separated media (e.g. teletype, voice' radio, etc.) whereThe par^anti 

. successfully- communicated, despite the lack of thes^ther, rich'sourcesbof infocmatioh. 

We currently^ have several hundred transcripts -of/ digjogues from a wide variety.of 
Sources. TheseJnclude; ' . * ^ "J - " ' 

1) Welp-seeKing dialogues fropi the TENEX timt-sharing System: 

2) Astronaut -and-'Grdund-Caritrol dialogues from the Apollo-13 
mission, • , 

3) . Tutoring transcripts from various sources. ■ ' _ ' 
• 4) Transcripts of radio talk sho^ 

We have - been using some of these to pretest the directions to the Observers. Some of 

these transcripts fail pur criteria in one or more ways, often because of occasioV>al 

intrusions by a third party.'*' We are alsq.' developing a facility to gather transcripts 

ourselves, directly froln a terminal-to-termlnat d^^^ ^, • 

Since there are certain phenomena we definitely do not plan-to model, and since we 
are committed to giving precisely the same 'transcript to the model as to the Observer, 
we will perform a certain amount of "cleaning up", of the transcript to get rid of the 
features which we know the model will not be able to deal with. In particular, we are 
not ■ modeling the participants' ability to : deal' with mispelings, abbrev'ns," and 
wordsruntogether, the various ways in which "a typist (especially on-line to a computer) 
will indicate local correx\Xctin\Npns, eTc. These and ^similar \inds of "noise" will be 



cornected before either th© model or the Observer see them. What wiH not be cleaned 

. ..... . . . ° . ■ ■;• ■ , ■ ■ 

Up is the anomalous grammar, inappropriate choice of words, saying things which clearly 
(to tis) weren't meant, 'arid the^ssorted fumblings with words ,w|irch happen when4he 
speaker can't, fijid^the riight wjyjT sxpress him§^^ * - 

Additional details, of^how a transcript is prepared for the Observer are given in the 
App'fendix, with examples. , 



Observer Commetttaries 

'Haying chosen a particular dialogue foi* study, we next give i^^ to an Observer, 
chosen from outside our research team, to bbtafn his com^mentary on the dialogue. Jhis . 
activity is descril^ed in detail in sections 5 through 12 of the report. (Thus the report 
. covers methods fbr a part of the experiment <5yde rather than the entire cycle.) 

Dialogue Process Models „ 

With both the transcript and the Observer's annotations to guide us, the next step is 
to build a set of processes vD-fiich will maintain a pair of simulated memorie? (of the 
informatiorj states of the two pjarticipants) based on the transcript as input. Although 
we will feel f?ee to reuse processes from the' modt^s- in our previous experiments when 
convenient, our intentions are that each of these*models be a one-of-a-kind. program, 
responsible only for the one "dialogue and one Observer on which it was based. This 
view contrasts with the more conventional approach of building a single system which, 
with each iteration of the experimental cycle, would b6 expected to adequately model 
the new dialogue/commentary pair os well as a\\ the preceding'ojies. 
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Obviously, certaiti supporting processes will be needed, regardless of the choice of 
transcript. Among these might be processes for: - ^ . 



1. Natural language parsing 

2. Semantic memory manage^ment 

3. Inference // 

i4. Discrepancy^ detection and resolution 

f : , ' - 5. %pothesfs(7geheration and testing - j 

^ J. 6. Evidence 6fvaluation < 

7. Awarehe^js of^time passage in dialogue 

i * 8* Attelatior^Vfocus ^ • ^ 

V 9^ Splectiv^forgetting ; : . / / 

V ? 10. Tim^ anjd spac6'resour<:e allocation ; , . • 
V - • ^--Extensive trac^ 

' Theffe p|pspective supporting processes sh^e common technical histories.. Each 
. J Pi'eced^hts in existing computer progrgms^ Each 

r one is .subject to simRiificatibn (relative to p^^st instances) because our models are case 
^odels/Zrather than general systems. For each, there exist adeeiuate technology and, 
; I personal skills for building the necessary experimental programs. Nearly all of this 

' j^.- . : expertise comes from artificial intelligence, cognitive psychology, and computational 
: ' . linguistics. " 



;■ |n|any model of a single dialogue, there is an" issue of what parts of the model 'are 
noijls/gnificant . because they are |3"h^^^^^ This problem. is addressed explicitly at the 



/■ ■• multiple experiment level. 
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Comparison ol^Process Behavior with Ohservei^, Commemnry 

Once have constructed the mofiel we wilj conduct an extensive comparison of the 
behavior of the; model with the Observer's coJrHentary. We expect to ident 
sets of results from each experiment: 



ify^ Thrfee 



'1:1 



W Program' states and actions ^|)rresponding to the Observer's commentary. 

Z . P4ienQmena' recorded in »ifie Observer's • comments ,for which there 
* :correspondin& states in fhe^-program. ^ / 



are no 



U ■ I ' " . ■ 4-""" 

3. Program sj^t^ which app^j^r to* contradict the Observerf's commentary. ] 

\ . . . • . /- ■■■ ■ 'V ■ 1 



]h addition, for tho§e aspects of the model which bear a positive rel^tfo^^ to the 
^ObserverX commentary/ we/will estimate the (actu&l or potential), generalityj of the 
'methods involved. v ^ I V^!^ 

F6r behavior noted by the Qbs^ver.but contradicted (or ignored) by the modet, we 
• ^'*ij^3"3'y2e why it was difficult or undesirabfe to^^chieve accuracy in the model ^tong' 
thefee dimensions. ^ ' ^ ; 



MqLTI-EXPERI M Em PROCESS ' \r j 

i As we repeat the .above experiment on new dialogues with new Qbs^rvers,^ 
e^^pect to accumulate a collection of processes which are reusable as is, or with minor 
generalization. Those jDrpcesses which continue to prove usefui over a c'ollectiorji jbf 
experiments are the ones which we will consider candidates for export. (See Figure 
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This is the answer to Ihe problem of adhocn6ss7 Processes are not identified in 
this way unless they continye to work over a range of dialogues. It is a conseU^Hve 
answer, in that some proce^es that In fact^are generllly applicable may nbt be 
e)jercised enough to 'ba r^je^ized iri this step. The rar|ge' of cases over which a- 



process hps been tested" is a|way^ explicit, so that the Widence"for its generalitylis 



clear. 
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One of fhe advantages tof 'this r^^ultistage 'approach i$ that ^we expect certain 
processes to prove themselves useful eaHy and thus' to provide ^us witl;i exportable 

results well before the end of the project. - ' 

. • , "« • ^ . * 

■ \ . , . -*■'.. ' ■ : 

DISSEMINATION OF DISaWEREp /ILGORITII MS " 

Dis^eminatioh of pur results is uniquely important for this projects ■ By the ver^ 
''nature of our goals* our Work is- broadly interdiscipiinary, with implications for computer 

science, psychology, linguistics, and otNf disciplines. The technical dissemination will 
• include the novel features of the methodology and the novel reUlt forms as well as 

specific processes. ^ , 

. ■ - ' ' - « . , ' ■ • ^- ' • . ■ 

Two kinds of dissemfnation of results are planned: one'conventional and one specific" 
to this method The first kind, of dissemination will occur through ^the usual scientific 
channels: site visits, scientific and.^jopular publication, conference attendance and 
presentatipns, as well as exchange of programs and, perhaps^ personne^. When we 
have established a collection of tepeatedly effective processes/ we plan to disseminate 
these^ results by actually retrofitting ihe processes to already existing man-machipe 
interfaces. Of course, they can be designed into new systems as well. 
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3' UNDERSTANDING THE OBSEHVEK'lS TASK 

1 . ' ■ /I ■ , ■ 

Two kinds 0/ input information are avail'abl^. tb the 0bserv6r as indicated in R 
He receives a dialogue transcript and a set of instructions on how to annotate ^ , [ 
He wiH also receivef training and practice in t^e u^e of these instructions. 



, The insti^uctions include a getierai orj|fntatiort. to the Experimental goals, i the Ose^ of 
the- Observej-'s work, and the ground rules for making judgments; they ^Iso include 
directions for several specific annotation tasks to be performed on. copfes ^f the / 
transcript. These tasks are performable indeperident of each other except for a fe;y 
specific sequence requirements. These anhbtations-^re 4he only output produced by 
• the Observer, ' *" 

' ■ - ■ ' . 

The task is a bidnd of specified, definite steps and pers<«>al, subjecfive judgments.. 
For example, we' ask the Observer to tell where eabh participant V ej^pressed 
comprehension of the other's remarks, but we do not tell him turn he is to kKbw this 

■ . . ■ ■ . ' ■■■ \'-' 

The subjective parts of the Observer's Work are essential because they engag^is 
eommunic^ation abilities and methods. Certain methods are used in both his own 
•performance of his task and in the dialogue participants' performance. It is these- 
methods. involving receptive acts in communication, tfet Will be represented in our 
models and their comparative evaluation. 

Q ■ 

The Skills used by the Observer are priririly those of an ordinary 
native-communlb^ in the language of the ' dialogu| b^g^ Observed (which always 
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Didlogue Transcript 
(muftiple copies) 



Annotated 
Transcripts 



Frgure 3-1 



The Qbserver^^S"a^k 
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happens to be English). One Observer is used for all of \he categories; tiie annotation 
task is not split across Observers, and it does not tncli^ pooled final judgments. Tliis 
assures tiiat theyiew of a dialogue tiiat getsbuilt into a model be as coiierent as 
the individual Observer vias. (It avoids models resembling committee reports.) The 

if ; I ■ . ■ • ■ • . V " ■ •;- . . 

Observer makes' judgments tijat fo him are obvious, clear case judgments/ so that tloe 

■J • .. ■ ■ ■ ■ - - ; , . . 

adequacy of the evidence in the transcripf is thereby assured. His work does not 
require sophisticated- technical understandii^g, of language, and ik does not include 

connputer-relate(4^ notation in any way. (In strict experimentation tVe Observer, tlie 

. ' \ 

model builders^ and the dialojgue participants are always separate.) We Vpect that his- 

j&dgments are representative of those made by the great \ajority of 

native-communicators in the course Of their actual comm^^ „ • 



The Observer will be supplied with information on the situation, roles, jargor 



medium of the dialogue. The general rule is that his access to each turn of the dialogue 
should be essentially as good as that of the participants. 

Our policy is not tMe^eive the Observer in^any^^I^B^ide our preferenc'^ for 
this approach on moral grounds, it has a number of-technieal^enefits as well, we 
believe that .the Observer's performance will be more stafble if he is not attem'pting t^ 
discover the "real" experiment. We also expect to disseminate the tichftology to 
interested volunteer Observers, which requires providing them- with the best possible 
access to it. Also specialists who may serve as Observers from time to time. Finally, 
effective deceit is difficult to achieve, and experiments which rely on it are inherently 



4. "PHENOMENA FOR 'OBSERVATION 



THEOBSERV/ITIONS ^ 



■ '('• 



We bave developed and pretested instructions for our Observers'^ to cbmment on a 

number of interesting- dialogue phenomena. /These include recognition , of references 

both to previously expressed conc0pts and to previous text. We also have Instructions 

fdr certain common dialogue forms, such as questions and commands, expression of 

comprehension (and confusion), and the correction of errors arhd misunderstandings. We 

fiaye a number of ways fo capture aspects of the content conveyed in a dialogue^ and 

the knowledge relied upon to comprehend this content. ^ This includes ways^ of 

g^ner^ting : and* judging paraphrases of dialogue utterances that permit a- detailed 

examination of the role of context in dialogue. In addition, we* have developed 

^ . . . ■ ■ (■ ■ . . ' 

instructions for Observers to annotate the topic structure in dialogue as introduced and 

X • . . . ■ ' / - . • ' 

terminated by each participant. 

CRITERIA FOR SELECTING PIIENOMEN/l . 

■ ■ ■ '■ V ■ ■ - ' ' ' V 

The" various phenomena described briefly above (and in detail below) by no means 
encompass all aspects of dialogue. There are a number of criteria that w^ have 
informally used to select phenomena to observe. ■ 



/ ■ . . . . ■ 

.B . ■ ■ ■ ■ 



1) Importance: 

^■'■''^ First of all, havb* considered only phenomena that we felt were ah 
important component in dialogue. Each set of instructions sheds* light on an 
aspect of dialogue th^t we feel will be central to an interesting model of 
twa-p^rty cfom.munication. ' • 

We have rec|uired that phenomena be central, in the sense that the 
conrfmunication would break down or be significantly changed in character if 
the phenomenon in c^uestion were eli'minated, ' Phenomena' related; to a 
particular dialogue source or context were not chosen, and phenomena which 
appear substantially unaltered in gjonologues were not chqsen, ' 

2) Clarity: J- - , 'U;-': ' 

In the course of writing these instructipfis, we developed and modified our 
concepts of Ihe phenomena, since we- wef e forced to specify in some detail 
what we wanted to observe. The original five c$tegories of phenomena to 
observe, specified in the original description (^f ouKwork (Mann, 1974) were 
developed into the present nine fcategori§s through this classification'process; 
The instructions* in this report have typically been through three or four major 
revisions to improve clarity. "Some of the phenomena that we consider 
important will not be included, in ijrimediately forthcoming experiments because 
we have not so far been able to, write clear instructions fgr their observation. 

' 3) Reliability! ' . 

Finally, the clear notions of what to observe had to Ije translated into 
instructions that produced reasonably consistent observations across 
Observers in our pretests. Our informal feeling of the present level of 
reliability can>e exjifessed as follows: Qne could define appropriate measures 
. of agreement on these annotations, fairly and reasonably, with whiqh the 
authors would score above 90 pferd^nt on most of these categories, working 
on the kind of dialogues involved irftVe pretest. ^ 



Th)s Development of Ob$ervation Categories 

In' the process of developing the nine categories presented in this report, 
examined fourteen othjsr observation categories in detail. 
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Three of these otHer categories were cfropp4?J because they failed one pf the 
criteria described above., ' This, was usually because the concepts could not be' 
operationaliied into coding rules for Observers. 



In many other cases, we,we>€Oable to shift from Initially unusable categories to 
others that bore on thersame phenomena in a way \yhich met oUr constraints. Seven 

: ■ ■ . : ' ■ , . « 

.categones developed into other categories in this way. * 1. ' 

. " ^ . ■ ' ■ ^ ' ' ' " >| 

» ■ ■• *■ 

< ' ■ , •■ ■ ■ •$ ' 

finally, four categories of observation are still in a state of development. These"" 
categories are promising, but have not yet developed to the stage where thej can be 
presented here. These, include observations of ceremonies that occur in dialogues, of 
comments about the dialogue itself, of the speaker's funptions (or purposes) in^ 
generating his utterances, and of defining events in which the rfTeanir^s of new terms or 
concepts are explicitly given, \ ^ V' * J 



VSo the set of observatron categories described in this report are those members of 
a larger set that survived the criteria above. We would expect to add to this set in the 
future after experien1:e with modeling and observation brings new aspects of dialogue 
to light. / - 

POST-PROCESSING 

As described previously, the commentary of the. Observers will be used to test the 
dialOgi|e models we build. This makes it necessary to compare the commentary to the 
behavior of the model. In some cases, this comparison is straightforward. For example, 
when the Observer annotates that two sets of words refer to the°same concept, it will 



be easy to tell whether ^ model also creates this correspondence. In either Uases;' 
some post-processing of the^Observer's ^annotatibn will b6 required before v^e can make 
a cbmparison; For example, we have derived' a simple algorithm for'sjSecilying the 

lareas in which a topicM^ shared by both participants, given the annotation of tbifio 

^- ,;-V',tJ' ■ ■■ . ■ -^Jl'---'^-^- ■ 

iriitiatlon and termination for ^ach participant separately. We^petft to;xi|^vejbp fiJHher 

post-processing, algorithms to Me us tc> test parts of the: model forfwfi% we have 
no direct observation methods;,! ' ^ V ' 



FORMAT OF THE INDIVIDUAL INSTRUCTIONS SBmONS^ - 

The next ' sections will discuss the instructions -d^^^ For ' each sety we 

will present the phenpmena to be observed, the instruction$ themselves, and an example 

■ ■' ' ■ . - ' ■ ' ' *■,"■""' ' ■ ■ 

of applying the instructions to a real dialogue. ^ ' r . 

The dialogue used for each example is an interaction between a computer user and 
the computer operator, conducted remotely via computer • terminals. . Thus the 
transcript, which is given in fulTin the Appendix, 'captures almost all the interaction 
(only the timing specifications are lacking). • 



( 



5. OHIENT/ITION FOR TuiojlSE^ER 



a- 



» I/, , 



. ■ The Observers' directions are presented; in Several sections, one for each.pf the ' ; 
classes of phenomena we are' Interested i a s J^lf'^et oiV ' '-^ 

in'structions which, precede and> are common to each of these sectfofis. For \QisS^oi 
brevity these are presented here,- once, rather thaYat the hpad|f edch separatje set. 



pBSEHVER'S GENERAL INSTRUCriOm 



We are interested in your colrtmentary on those featurjfes ^ncJ phenpmena 
^ which ypu regard as clear and obvioys. Do not worry abot|jt making subtle or 
quesfibhable judgments. ^..Fpr those instruction fofm: -identify those 

parts of the dralpgue hjaving the property ../ 'V ^r ypM 'W 
whether a certain; segment meets the criteria se^ forth in Ithe directions,* just 
ignore it. Some instructioris will have the form: Tor' piece of dialogue 
. already identified, classify it into one of the following categories:. ... ^ th0se 
sets of categories will always have one labejied: "other/unknowji/unsur'e"; iise 
this classification freely if you aren't feure thit'one of the other classifications 
is appropriate. . v/ " 

Some of the transcripts we are wbrking^^^^^ 4 
secretary, listening to ^ recorded dialogue; others have lD^n typed by the 
participants the?nsefyes as they were conducting the dialogue via terminals.' 
You will be told which of these cases applies to the dialogue you are 
annotating. This is only -important to, you in one regard: punptuatioti. If the 
transcript was prepared by anyone 'other tH^ii' t^^^^ actual participant in the 
dialogue, then the punctuation you see is to be taken as only, a.^!good guess," 
possibly in error. On the other hand, if th|;^ranscript was created by the 
participants^ typing, then the punctuation can be taken to fee- what ^t^ 
participant actually expressed, with no editorial interpretafion invQived. 

Alfeugh a few of the directions ask you tp differenti'aite between the 
knowledge of the two participants,^ most do npt, since it is assumed that both 
participants understood the conversation in ^substantially the same wa^. In 
the case where the instructior^ clearly pre$ume:^that the participants share an 
understanding of what is being comrhunic^ted and you perceive that any of 
your annotations reflect only the unflerstanding of one of the participants," 
indicate this by adding the name (letter) of that participant to the annotation. 
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A number of the directions call for you to indicate a certain segment of 
the dialogue having some property. You may find th§f the segment has sharp 
Isoundaries, or tjiey lmay be somewhat indefinite. For all these' directions 
which call. for you to mark these segment Itoundaries (with what we will call 
segment markers") the following general instruMions apply: ^ 

Put a "[" at the pofnrwhere a segment abruptly starts, and put a 'T at the 
point where a segment suddenly ends. 

Graphically: - ' ' 

" I p 

A: tex^ text text [.text text text 

■ • , , - .. . ■ « . 

««««««<<< I »»»»»»>> ■ 

! (definitely (.(definitely 

• NOT part , I IS part • 

of segment) I of segment) ■ 

. ■ ' . ', 

After you finish marking these abrupt- boundaries, go back to the 
beginning of the dialogue and determine where gradual starts and ends -occur. 
Mark a "(" at the first poir^that a segment is clearly under way, and g "(=" at 
the latest point (prior to ffle "(" ) that clearly is not included in .the segment. 
Similarly for gradual segment ends, mark a ")" at the Mast point that the' 
segment is clearly under way, and a "=)" at the first point (following the ")" ) 
that the segment clearly "does not include. * 

*^(3raphically: \ ' ^ 

B: text text text te1<t ) text text text text =) text text text 

<«<«<<«««<««< I I »»»»>»>»>»»> 

(definitely IS part I (unclear whether .1 (definitelu not 

of segment) \ or not is part I part of ,segjnena -=^ 

I of segment) I '^■'^ s-'p 

Notice that each use of "[" is equivalent to a simultaneous occurrence of ' 
"(-'>hd T. , " • ; ^ 

■"^ . ' ' 

Several of the instructions will call for you to cfeate labels fof segments 
of Jhe dialogue; these labels will consist of One or more letters followed by a" 
number. This choice of numb^rsJs completely^ arbitrary you need not'^'be 
concerned with keeping them either consecutive or in order (or corresponding 
to the numbers used In. compliance with other sets of directions). 



^7 
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Whenever you feel that the actual annotations are not capturing the 
phenomena to which they are addressed, please noteJhis and add your own 
comments at any point. 
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6. REPEATED REFERENCE AND TEXT REFERENCE 

■ • ' . // .■■ ' . ^ ■ , . '] 

One of the most common phenomena in language is that of reference, the use of a 
sequence of words by a speaker to refer to an object br concept Taken as a whole, 
the concept of- reference is extremely ^omglicsted, encompassing as it does almost the 
entire range of human conceptual and experiential abiiities. Not surprisingly, we do not 
aspire to model the full range of reference" phenomena in natural language.' 

We have made two major restrictions on the phenomena we are investigating: First, 
we aj-e only interested in those cases .for .which the same object or concept is referred 
to by a participant in more than ope place in the dialogue. Second, we do not\ltempt 
to pin down exactly what is being referred to4rather, we simply want to^-defermine 
which sequences of words have been used to refer to the same "thing (whatever it may 
be). ^ * . , . . 

\ Within these boundaries, we investigate a few of different flavors Of reference: 

' f / - " - . * ' 

L Two regions of dialogue refer to the same thing. 

2. One region refers to a set of things and a second region refers to a single thing 
which is a member of that set, ' 
• 3. One region refers to a set of things, a subset of which is referred to by the 
second region. 

^^^Ate^re^'also investigating a form of reference with a somewhat different thrust: a 
so-called "T^xt Reference," made tcf a string of words in the preceding dialogue itself ' 
(and not to the referent of that preceding string of words!). 
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OBSERVER'S DIRECTIONS FOR 
REPE/ITED REFERENCE AN.D TEXT REFERENCE 

Repeated Reference 

\ A Repeated Reference is said to occur whenever trVo |hrases in a dialogue 
are used to refer to the same thing (object, person, activity, concept, ...). 
Some exahjples of such Repeated References are the following: ' 

• 1. . Repeated Reference to an object , . ' 

'A: I really like your wrist watch, • 
B: It only cost S50,00. 
? Repeated Reference to a person: 

A; 1 spoke to Max's sister yesterday. ^ ' 

B: Did you know that she's going to Europe » . 

next month? / . : * 

3. Repeated Reference to an activity 

A: Sky diving isf a great sport.' . . - 

' ' Bt Yes, and it's not as dangerous as some 

people think. ' 

4. Repeated Reference to a concept 

A: Jim told me that jobs are v^ry hard 'to 
fmd now. * , \, 
B:. Yes, that's true* It certainly 
/ is sad„. 



The two phrases may be identical in wording, but in general are not A 
Rapaaied Reference may be effected by as little as a single word, or as muph 
as a .complete utterance, A phrase may participate in many Repeated 
References or in none, - 

Whenever in your judgment two phrased are obviously used to refAr to 
the same thing ol) underline. each phrase, 2) assign the first a unique; number, 
3)» write this number in either margin, opposite each of the two phrases, A) 
connect each phrasQ to the number you just wrote with a line. 

U: ls [NAMES] still around? 

- .\ . . ; r —(i) 

0: No' he isn't. * . ' 

a)~T 

If the same thing is referred to by means of more than two phrases, ..ise 
the number assigned to the first such phrase in annotating each subsequent 
Bepggtgd Ref erence. 



U: Can you recover those files for me? 

(7)- — ^ 

0: OK hold tfn just a min. and I'll try to find them 

X 



(7) 



0: OK yes I have found the files you wore concerned abou f 

17) ^ ■ . . ■ " 

You may discover *hat one phrase is used to refer to a single element of* 
the ^et or collection of things referred to elsewhere by means of^ a second 
phrase. Whenever this happen^;^ annotate these as Repeated References with 
the addition of an or "S'Mr^spectively) on the line from the Reference to 
an element or to a set. For example: 

0: ^ r - am really not sure, however I know that that would 



not be responsibility /we would take on ourselves ,„ ^. 

ii)s— — - T TT 

Sometimes one may find that one phrase is used to refer to a set and a 
second phrase is used to refer,'not to^ an element, but to a subset of the set 
referred to by ^eans of the first 'phrase. In such 'cases of Repeated 
References, mark the Reference to the^set with arf "S" (as requested above), 
and mark the Reference to the subset with *;SS". For example: 

U: Are the files that are on disk . archived every week? 

Qv'Yes. ^ / ' 

U: I had some files in one day and the next they ' were gone. 

:■ ■ "t; : . 

In the case of one Reference within another, underlirfe and bracket Jhe 
larger one? overline and bracket the smaller one. Forlexample: " 



o 

ERIC 



U: iQne ofi my files! | is missing. 

(3)e } 

J 0: Was jt marKed to be archived? 



U: i^ol none of ^/ihenrl 



— S(3) 



ssid) 



Referring expres^jW are a very copimon phenomenon in languageAso 
much so that some are ^easily overlooked To help you be sure that you are 
< considering all possible Repealed iWfTKOHci?*,'^ pay particular attent 
phrases beginning with \^ ' 

1. Personal Pronouns V * ^ ^ - 
(I you he she it we they, me him her us them) N.a: the^ will be 
one-word phrases. • ? 

2. Quantifiers 
(a an the that each all some any every one ...) 

3. Pronominal f^ossesstVes 

(my your his her our its their mine yours hers ours theirs) UB,: 
these phrases may have two Repeated Refcrencar. one for the 
possessor and one for the object possessed 

i ■ - ' 

4. Wh-words • 

(who what whfen wfiere why which how ...) 

f ■ . ■ 

These words are a intended to serve as clues to most of the potential 

Repeated References but are not intended to be comprehensive. ANY two sets 

Of words which ybu feel refer to the same thing are to be treated as a 

% Repeated Reference, 



Since they are so common in 
first and secon<^persbn pronouns. 



li^logue, we are making, a special case of 
occurrences of first and seconc^ person 



SINGULAR pronouns {I me my mln? ^our yours), when used to refer to self 
and partne^^ xespectivejy./ are simjoly ' to be underlined/with no further 
annotation necessary. Should you find an occurrence of a plural pronoun in 



the first oi^d^nd person, select one Iconvenient occurrence of a singular 

pronoun /(jsed by the same speaker,, in t4 same person. Annotate these two 

• (the occurrences of the singular and plural pronouns) as per the instructions 

for. general "element" afid "set" References, above. Note {hat first ana secQnd 

' '. ' ' ^ Si- 

person singular possessive pronouns jjiay st'ill requird annoitation as.,.a 

Reference to the 'thing possessed, but npt as a Refbrence to theWssessor 

You should be careful to distinguish (where possible) between "you" 
-referring to the speaker's partner and "you" referring to an'indefinite "other" 
(e.g., as in "You can't fight City Hail"). When you come across a £lear instance 
of .an impersonal or general "you", "do not. underline itrr'ather, indicate the 
occurrence of an impersonal "you" by circling it in the dialogue. If you are ' 
not sure whether or not a particular occurrence of "you" is impersonal, simply 
do not annotate it at all - neither underline it nor circle it. ' 

• ■ ^* ■ 

TaxiRaferanca * 

A'^Text Reference occurs whenever reference is made. |o previously 
occurring words within the transcript. For example, in the sentence 

Go 3 blocks and turn North; 

By North I mean towards the mountains. 

the second use, of "Worth" is a Text Reference to the first. We call this a 
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Toa^l Rafdrmea because it refers to the previous use of the word itself, rather 
than to its meaning. " ^ 



Many of what we are ^calling Text References would be indicated, in a 

. , ■ " M . • " ' ^ 

formal publication, by quotation irtaicKs/ However, the transcripts typically will 

■ - • ^'"^ > ^ ■ . 

not include these quotation marKs. Note t'Hat a Tex i References need not 

repeat the words to which reference is beingNrtade. For example: 
In your last thr^e sentences, you failed to answer.me. 



Mark Text Referenies in thj^ame manner as Repealed References , 
and distinguish them with a "TR" next to the number. For example: * 

U: The names- are )[file namel3.dat;l < ABDLLLL^ / check that 
[file name'13.idat;l,2 [file name2].f4;l and [file namr3].f4;l 

1 ■ V . ■■ ■ 

0: On the first oj:te"l assume the ABDLLLL was an *rror right? 

U: Right _that should have been [file namej-dat; 1 and 2. ° 

Som^imes tfiere will be a Text Reference to all or one of a member of 
separate pieces of text, each involving the same phrase or word You need 
annotate only the most recent, prior occurrence of the text which seems to 
have the same meaning. For example: 



/ (.7) 




E: An interpreter is al^s faster than a compiler. 



A: Why? 

t: An interpreter starts producing output immediately. 



- - ^ ^ _ . . , . I 11 i>wi et er ? 



Ar What do you mean by an interpreter ? 



Be sure you examine all occurrences of: 

* SAY ' 
MEAN . ■ 
MENTION 
TALK ABOUT 

DEFINE - - ^ 

-•v. , . " ■ ^ , ■ • - ■ . ■ . . V 

and similar phrases (and their other tenses) to see whether they signal the 
presence of Text References. A$ before, this list is is only suggestive and is 
not presented^as being comprehensive. 

Figure 6"1 represents a portion of a dialogue fully annotated for Repented 
Reference, * ^ 
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• . 15 0: YES 



CO- 



.16 U: INfONE OF MY DIRECT0RIES)!_HAD SOME FILES IN ONE DAY AND THE NEXT THEY WERE 

■ 17 GONE AND THEY WERE NOT ARCHIVED " ' ^ — ~~ — ^ ^ ^ 



• ^ 18 0:WELL,_l_AM REALLY NOT THAT SURE OF HO'W THE ARCHIVEING OF ■ 

; 19 THE FILES ARE DONE SOJ_WOULD SUGGEST THAT YDU^SEND A MESSAGE TO 
\ 20 [name2] IN REGARDS TO TH^^ 

K Aa)- — • . - ■ 

• 21 DURING THE SWING SHIFT SO IN ORDER THAT YOU DO DON'T Not GET . 

22 MISLED BY SOMETHINGJ^AM NOT CERTAIN ON j WOULD RATHER 

• 23- YOU DISCUSS THIS WITH [name2]. HE IS OUT OF HIS OFFICE AT THIS . . 

I o \ """ "^^ ^^"^ „')^COULD SEND HIM A MESSAGE ANDJ^AM SURE HE WILL GIVE 

^ ^ 25 YOU ALL THE INFORMATION YOU^WOULD LIKE, [operator's name] 

- CV)S -ygS U; OK [operator's n^nej. CAN YOU PEOPLE/gO INTO ANY .DIRECTORY AND DELETE FILES WHEN YOU 



C 7) -^IL ^ I^OTICE THAj ^EY ARE NOT BEING USEP) [tiser^ n.-.mp] ' \ -^CV) 



(2) 



{S)e ff_^^'^ REALLY NOT SURE, HOWEVER KNOW THAT THAT WOULD NOT BE ' 

^ 29 RESPONSIBILITY WE WOULD TAKE IjPON OURSELVES^ TO JUDGE WHETHER OR ^ 
■ ' 30 NOT 'you want YOUR FILES OR NoV. '■ "/ / 



— C7) 



31 [outside interruption here] - : . 

32 0: [user> name], ARE YOU THERE? 

Figure 6-la - Dialogue Annotated for Repealed Reference '- Part 
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. 33 LINK FROM [user's name], JOB 21, TTY 22 " • 

'. ■■. - ■ V . ^' ■ ■ 

34 U: SORRY [operator's name] > • — ^- — L«f^V) 

35 W E GOT DISCONNECTED....CAN Y gy RE(| oVER THOSE FILES FOR M^^' 1 €(W 

(0) FAR ,ASj,KNOW THEY WERE-I/THE DIRECTORY ON THE 16TH... THE NAMES ^ ~ 

.37 ARE ...[file namel].DAT;lABDLLLL CHECK THAT . ..[file name°l].DAT;1.2^ ^ . '" ^P^ 

38 [file name2].F4il ....ANQ ^Cfilp "ame3].F4il. .... [user's name] ' ^ T 

39 0: OK HOLD ON JUST A MINUTE AND I WILL TRY TO' FIND THEM • 

• . ^ . > ■ I- , , ' ■ - . . 

Figure "Sr^lb Dialogue Annotatekior Repeated Reference - Part b 
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7. REQUESTS ^' 



/ In the course of a dialogue, frequently one of the participants will communicate to 
his partner an expectation about the partn&r^s subsequent' behavior. We intend to 
mtlude alt such communications under the heading of Requests The Observer is asked 
to detect any occurrence of a speaker^ indicating such an expectation, 'and to .classify 
each of these utterances into one of five categories: * 



1. ^ Questions — request immediate, verbal response. 

2. Orders — request immediate, nonverbal response. 

3. Directives — request.certain behavior in the future. 

4. Rhetoricals — lobk Wke Requests but are not; - 

5. Prohibitives — request TO NOT 00 something 

Having done this, we asK the Observer to annotate the; immediately^ following verbal 
response by the requester's partner. These annotations attempt to characterize the 
cooperative or uncooperative nature of this response. 



OBSERVliR'S INSTRUCTIONS FOR REQUESTS ^ 

We are interested in the general category of utterances ^which we call 
RequestSf by which a speaker communicates a specific expectation, request, or 
demand to the hearer. We are also interested in the cooperative . or 

.uncooperative nature of the hearer's .next reply. 

J ' • ^ 

X ' ■ • .. ' ' 

There are many different ways a speaker may express a Request: 
direct: "Please pass the salt." 

indirect; 'T could sure use some salt." 

The speaker may Rcqui^st behavior which is; 

"Is MTAO write-enabled?" 
"Please mount JMlla on DTA3." 
"Do you have the time to retrieve my 
file X from archives?" 



verbal: 

nonverbal; 

both: 




The behavior Requested may be: i 

jmmediate: . • "Attach the pump to the platform.'* 
delayed: "Please calL me when you arriv^-" 

There is a set of utterances which look like Requests, but which bdth 
speaker and hearer know are not. The 'following are examples of this 
• category of Requests: . • • 

"Why don't you go jump in the' lake?" ^ .^ ■ ' 

. "Who do^ou think you are?" ) ' ' . 

"Is the sky blue?" . , " ' ° • - 

"Go fight City Hall!" , 

. Finally, there are Requests* to not do Oh say Something: - 

"Don't think about elephants." ^ ; ' 

"Whatever you do, please, don't throw irie into the bri^V patch!**^ 
"Don't put beans in your. ears." 

"Never low your car while it is in gear." \ , - 

We divide the set of possibfaf^equests into the following groups: 

1. QUESTIbNS- For our purposes, "Question" refers to a mucKJarger 
class of utterances than does the conventional, casual use pf the word. ^By 
^"QuestiDn" we mean any utterance by which the speaker communic;^ites to the 
hearer an expectation,, request, or demand for specific, immediate, verbal 
behavior. / 
■ ■ ■ ■" ■ ^ ■ ' '■ ■ ■ ■"■ ■ ' ' ■ ■ " ^ ■ U 
Verbal: Xhe anticipated response is an utterance. . ^ 

Irvimediate: The hearer is td initiate bis compliance commencing with his 
verj'. next speech act. ./ 

v' • 1 ■ . 

Specific:- The expression is ^suff icierrtly detailed that it is potentiaTfy 
fulfillable by a single, appropriate response. ("How do I take a square fbof" 
is specific; "Help me, I'm lost!" is nob) • 



2. . ORDERS. By "Order" we mean any utterance by which the speaker 
communicates to the hearer an expectation, request, or demand for specific, 
immediate, nonverbal behavior. « " ■ 

Immediate:, The requested nonverbal behavior is to commence as soon as 
the hearer completes the process of comprehending the order. . . ° 

I' . r T ' , ' 

Specific: The speaker communicate.s the belief that with this Order (and 
what preceded it) the hearer has sufficient details to enable him to perform 
the desired behavior. , ~ 



■ 3^ DIRECTIVES; , By "Directrve" we mean an utterance by which the' 
speaker communicate^^ to the hearer an expectation, request, or demand for 
any sort of /)fl/ayhd^ fiWjflw'or. : 




■ . Delayed: the ipeaker communicates rfo sense af immediacy of respdnse>in 
his Request (ifiQluding, of course, Ihe-case where a future time or condltion\M!' 
indicated explicitly). . ' \ 

•: 4/ R^iefoRICALS." A "rhetorical" is an utterarl'e which has the form of a 
Request, _^fe{ut which, in the context of its use, is uru^erstood by all involved to 
represe^rt something other than a mandate to the hearer. ' ^ 

•..ia P'^O'^'BlTiyES. Any Request to not\ngafC(t in a particular behavior is a ' 
Prohibitive." Note that this need not- require a negative word ("Trespassers 
Will be eaten!"). This class also includes the "don't do anything-don't go 
away-rdon't be Impatient", implied by "Wait!" and its related forms. 

Frequently a Request' will be 'taken, by custom, to mean- something 
different from (but possibly related to) tlle. literal sense attributable to it. A 
simple example, of this is th^Xeqwest "Do you have a match?*^; Clearly, those 
cases in which this is asked as a'simple yes/no question would be' regarded by 
the average speaker in our culture as atypical. :iFor the purposes of the 
above classification, you will be a$ked to label a Request according to the 
principal force of , the utterance ~ the deafly intended and recognized 
meaning of the Request, independent of its surface fprm. ' In the case of the 
above example, "Do you have a match?" would be annotated as though it Were . 
"Please give me a match." 

Any Request which falls Into more than bne of these categories should be ' 
annotated separately for each appropriate category. 

Should you encounter an utterance which seems to satisl^; fhe general 
definitipn of Requests described above, but which doesn't fit into any of these p 
subcategories, then underline it and describe the new subcategory of which * 
you feel it is a merftber. 

Note that in nonpathoiagical dialogue, any utterance may create a general 
expectation of relevance, continuity, etc. 'in \the subsequent reply. The 
creation of this type of expectation is hot "intended to be included in our 
definition of Request. Thus, altjiough in the sequence: A: "I don't feel well/VB: 
"Santa Monica." B would not seem to have satisfied A's expedjations, we would 
not regard A as having uttered a Request. , \ " 

You are to annotate each of these ' <^ategoriesi bf Request separately, * 
iacQording to the following directions. ^-v - 
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Pm i —Quettiont. . 

■ ■ ■ ■ . ., • ; \ 

Whenever pu see a Question, enclose it in angle brackets/ mark it with^a 
, Q and a unique number. ' 

e,g.: ■ . . ■ ^ ■ ■ . . ' 

(,Ql ^Tell me your name^ ^ « 

Do not rely on the presence or absence of question 'marks in the 

transcript? Is that clear. . Remember that any contiguous regijbn of an 

utterance that exhibits ah expectation . of an immediate', speciitc, verbal 
response falls into the category of a Question. , * 

' Question so annotate^!, examine the immediately following turn of 

the otKpr person. We 'want you lo separate two classes of turns: a) the t'"-n 
provides (or begins/to provide) the anticipated behavior, and b) everything 
els6. .If this turn contains one or more utterances which constitute some or all 
Of/ the specifically requested verbal behavior, then annot.ate the turn with the 
QUestibn label and a te.g.: -Q5+"). Otherwise, annotate the "response with 
the same label and a (e.g.: "Q5-"). 



{ Note that the turn may begin with verbal behavior outside that requested,/ 
( Hmmm, gee, let me think, I don't know, well," yes!, J guess so."). . As lorjgW 
the recipient of the Question begins to provide the requested behavior prio? 
to his partner's next turn, we want you to .annotate this as 'V*. Do not 
attempt to annotate the intervening behavior with respect to this particular 
Question. 



Be sure you distinguish between those responses which actually pri/vide 
the desired behavior and those which (only) setjn motion a chain of events 
which is expected to culminate in the desired behavior. An example of thi.s 
would be a response which sought to clarify the question, thus indicating'a 
willingness to answer it (but which was not itself initiating the answer). 

For those responses which are marked select one of the following 
descriptions which^'most closely captures the function performed by that 
response; append the description's label to the annotation (e.g.: *'Q5-A3"). If 
none seems to apply, then invent and describe your own. If the response 
serves more than one-of these functions, choose the one indicated first; if*this 
distinction is not clear, indicate all the apparently simultaneous responses (e.g.. 
"Q5-A1,A2"). - 



Al. Requests clarification of meaning of Question 
or definition of a term in the Question. 
..A: How old is your grandmother? 
B: Which one? 




» ♦ 



A2. Requests clarification of basis or cause for 

asking the Question. » 
A: Where were you last night? / 
B: Why do you ask? 

A3. Challenges a premise or assumption in the 
Question, 

. A: Why did you go to. the p'arty without me? 
. B: What makes you think I was there? 

(IMote that Al through A3 usually create new Questions.) 

A4. Indicates that he may have or know the 
information but refuses to supply it 
A: Where did you get the SI 0,000? 

B: I refuse to answer on the gfounds ,,. , ! 

A5. Dismisses Question as Waningless or 
otherwise unsuitable for answering, 

A: Do you think the union will ask for more 
money? 

B: I won't dignify that with a reply. 

A6, Exhibits a pron[tis^^ willingness or y intention 
to reply to the Question, somehow, after a delay. 
y , A: So what's the final^igure? 

B: I'll have that for .you this afternoon. 

' jt^ ' ' . , ■ • ' ' 

A7. Disclaims knowledge of the requested 
• information. ^ * 

A: Can you tell me anythmg about LINK? 
B: Sorry, I'm new here and don't know. 

• . • ' 

A8. Declines to supply the information. 
A: How much do you make?' 
B: Tha*?s none of your busihpss! 

A^. Does not take up the Question in any way. 

A: Young man, what are your intentions concerning 

my daughter? 
B: Hasn't the weather been lovely lately? 

Identify the region of the transcript which you used as the basis for your 
choice of labelling. Use double angle brackets, i.e., «, ». . Label each 
brackef with the Question identifier, e.g. {Q5). 
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After you have read the whole transcript and identified all of the (Q)'s, go 
back and decide, for each -one, whether the expected response was ,c«er 
supplied In the transcript. If so, identify the region of the transcript \n which 
It was supplied. Use the segment markers described in the general 
instructions, i.e., (-, (, [, ), Label eacjunarker with the Question identifier, 
^f' u 2 substantial sa^ (a sentence or more) in the region in 

which the response is supplied, usfe several sets of marked brackets rather 
. Id "r°Il^./^*V ^^'^ Qiiestion, label the rightmost ] or -), with the word 
Kartial if only part of the requested verbal behavior was exhibited. ' 

it -may appear that the participants end up with different views on 
wliether the requested behavior was supplied, or^whether it was supplied in 
full. If so, then label your brackets with the participant's identifier when 
indicating where the information was supplied, e.g., ](Q5 for George). Use 
two- sets of bracket-s if necessary. 

■ ' ■ ■. ; ■ ' ■ " 

Part 2 - Orders. C ■ ' / ' 

_ A contiguous region of an utterance that exhibits an expectation of an 
•immediate, specific, nonverbal response, falls -into the category of an Order. 
Whenever you see an Order, enclose it in angle brackets and mark it with "0" 
and a uniejue numbefi as above. - ~ . 

• ,' ■ • ■ , ' ' )■ .■ 

For -each Order so annotated, examine the "immediately following turn. We 
want you to separate two classes of responses: a) compliant and b) everything 
else. There are three kinds of compliant responses: ^ 

•.'*■■ J I . ' 

CI. Asserts completion of requested activity. 
A: Now attach the second bolt. ' . 
B: I've already attached all of them. 

C2. Asserts current performance of requested activity. * 
A: Next, empty the/humber two tank. 
B: I'm already dmrtg that. 

C3. Asserts willingness, ability, and intention * 
to initiate requested activity forthwith. 
A: Private, do you'^think you could find me a 
\ • cup^of coffee? • , 

6: Yes sir! Right away sir! 

• '. ■ ? ' 

vLabel the compliant response with the Order label and a V followed by 
the type of response (e.g.: ,"(05+02)"). If you feel that there has occurred a 
compliant response which, does not fall into one of the above categories, feel 
free to invent a new category of your own. For those responses which are 
other than compliant, annotate- them with the Order label and a "-", as above. 
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For those responses which are marked select one of the following 
descriptions which most closely captures the function -performed by that 
response; append the description's label to the Annotation (e.g.:"(05"R3)"). 
If none seems to apply, then invent and describe your own, 

RL '^Requests clarification of ^meaning of \, - 

Order, / . 

A: Position the ^rain towar^^s the front. ; . 

Bj Which side is the front? \ 

^ % . • ■ . • \^ • ■ 

' R2. Requests clarification of basis t)r cause for 
I giving the Order. ' • * . ^ . 

^ ... A: Glue the bottom before the top. 
• ^ " B: What will that buy us? ^ ' ' . ^ 

'•# . . ■ 

^ R3. Challenges a premise or assumption in the 
^ Order. ' ^ 

A: Set up a meeting with Jones this 
afternoon/ ^ 
, B: Have you forgotten? He's on vacation. 

(Note that Al through A3 usually create new Questions,) * 

^ R4. Indicates thatjj^ay be aBle to perform the 
Order but refuses to perform it. 

A: Loan me SlOO until payday, if you have it- ' - i, 
' B: I could find it, but I don't lend money to 
anyone. ^ 

^ R5. Dismisses Order as meaningless or otherwise 
unsuitable for performance, 

A; Will you give me a ride to the station? 
B: Sorry \ can't, my car's in the shop. 



R6. Exhibits a promise, willingness or intention 
to perform the Order after a delay. 
0 A: Johnny, go cut the lawn. 
, B: I'll do it this afternoon." 

R7- Disclaims ability to perform ther Order. . ' 
, ^ A: Please mount my tape on drive 1. 
, B: Sorry, I'm new don't know how. 

RSf Declines to perform the Order. 

At Subscribe now and save 20%! 

B: No thanks, I'm really not interested. 
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f?&. Does not take Up the Order in any way. ' ■ ■ -vv 

• As Point the camera towards the meter. • ♦ 

B: How many washers are there supposed to be? 

As above, identify the region of the transcript which you used as the basis 
for your choice of labelling. Use double angle brackets, \.e., «, y^. Label 
each bracket with the Order identifier, e.g., (05). 

... ..... . ' ' ^ 

L 1.^"^! ^^^^ read the whole transcript and identified all of the (0)'s, go 
back and decide, for each one, whether it was responded to 'anywhere else in 
Jhe transcript. If so, identify the region of the transcript which constituted 
the response Use the segment markers described in the general instructions, 
I.e., (, [, ], ), Label each marker .with the Order identif ier, #.g., (05). 

For each Order, label the rightmost ] or -), with the word "Partial" if only 
part of the requested behavior seems to have Been provided. If he tried but 
did not succeed, that's "Partial," too. 



Part 3 — Directives. 

\> , ■ . ■ ■ . ■ 

.Any contiguous region of an utterance which exhibits *^an expectation for 
specific-behavior in the future falls into the category 'of a Directive 
Whenever you see a Directive, enclose it with angle brackets, mark it with a ' 
D and a unique number, as above. • 

A Directive has the sense. "Perform behavior X at^some time, or under 
some circumstance, in the future." .Treat each Directive as though it said "I 
would like for you to perform behavior X ... . Do you assent/commit, now, 
to behaving as indicated, at'th^ appointed time or under the . requested 
circumstances?". -^^v ^ . 



Thus, a Directive combines the functions of a Question ("Tell me that you 
will ... "). with those of an Order '(" ... perform a certain behavior."). 'With 
this in mind, the response for each Directive is to be annotated according to 
the directi^sjor Questions (i.e., responding to a, request for an immediate, 
verbal gctt^Ttie exception to this is whenever the response is like the CI, 
.02, or C3 responses to Orders. Treat the requested action in the present 
tense, in this case, annotate as a "+" response to a Directive. 

As above,' identify the region of the transcript which -yOu used as the bisis 
for your choice of labelling. Use double, angle brackets, i.e., «, >>. Label 
each bracket with the Directive identifier, e.g., (D5). 

After you have read the whole transcript and identified all of the (D)4, go- 
t>ack and decide, for each one, whether it was responded to anywhere else in 
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the transcript. If so, identify the nigion of the transcript which constituted 
the response. Use the segment markers ^described in th^ general instructions, 
_\ i.^u-i „ _ -.-4 ^jji^ Directive identifier, e.g., (05).- 



i.e., («•, (,.[, ], ), »). Label each marf^if 



For each Directive, label the rightmost ] or «), with' the word Tartiar if 
only part of the requested behavior sfeems to have been provided. If he tried 
^ but did not succeed, that's Tartial", tok • ' 

I ' . 

Parts 4 and 5 — RKetoHcah and Prohil 

Whenever you see a Rhetorical br ,a Prohibitive, enclose ft in ahgle* 
brackets and Ynark it with ^ "R" or. T",lrespectively, and a unique number, 
above. Do not attempt to annotate the Wl&diate response to these Requests. 

After you have read Jhe whole tranWipt and identified all of the Fs and 
P's, go back angi decide, for each one, whether the corresponding (unexpected) 
behavior was ever indicated in th© transcript. If so, identify'the region of the 
transcript in which it indicated. Use the segmenf markers described in the 
general instructions, i,e, (=, (,*[, ],),-). Label o^ach marker with the 
Rhetorical or Prohibitive identifier, (e.g., R5 or P5)^ Jf there .are substantial 
gaps (a sentence Or more) in the .region in which the response is supplied, use 
several s^ls, of marked brackets rather than one set. 

Finals (ypjwrar instructions. 

If you observe any place where one participant seems clearly to 
° misunderstand his parlner's Request, underline the passages v^hich led you to 
conclude this and summarize, in .your own words, the nature of the 
misunderstanding. 



If you find a Request that sefemsMo you to be a repetition, perhaps with 
additional detail, of a previous question posed by the same speaker, indicate 
this by noting, next to the label for'the pev/ Requ^ast, followed b/ the old 
/?€?<7U€?Ai?.t label (e.g., "=Q5"). 

You should be particularly careful with utterances which appear to be 
Requests but which, in conte^ct, are only presented as descriptions of behavior 
rather than a ^quest to perform the behavior. For example: ' • 

A: How can I see^tter in here? 
' . B: Turn on your light) stupid! 

A: Oh, thanks. , ' 

The point of this example is that B is conveying to A the information he 
requested, not necessarily asking A to do anything at all. For these cases, the 
apparent "Wf?*7u<?st'V (by B) is simply to be Ignored. 

Figure 7-1 indicates a segment of dialogue annotated according to these 
directions. . . 



46 



■ ■ ■ ■ ■ • 

rCiXV 25 U:OK [operator's name]^CAN YOU PEOPLE GO INTO ANY DIRECTORY AND.,DE'L,ETE FILES WHEN. YOU 
^NOTICE THAT THEY ARE" NOT BEING USEd\ [user's r^mel 
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((3 1 28 ^ AM REALLY NOT SURE, HOWEVER I KNOW THAT THAT WOULD NOT BE 

. • 29 RESPONSIBILITY WE WOULD JA|(E UPON OURSELVES TO JUDGE WHETHER OR 
- 3P NOT YOU WANT YOUR FILES OR NOT.^^ 

/ . .... ^ . • 

. 31 ^[outside interruption here] . ' ' 

32 0: [user's nameJ^ARE YOU THERE?^ 



33 LINK FROM [uspr's name], JOB 21, TTY 22 

34 ■ U: SORRY [operator's name] . 
C^i ) 35 ■ WE GOT DISCONNECTEDi^CAN YOU ^OVER THOSE FILfS FOR ME..^ 

96 AS FAR AS I KNOW THEY?^ERE IN THE DIRECTORY ON THE 16TH..THE NAMES 

3/ ARE ...[file name IJDATslABDLLLL CHECK THAT ...[file name! ].DAT;1, 

. . 38 [file name2].F4jl....AND [file name3].F45l.....[user's name] 

(.0(7 AM Q^C°"V°'-°°'^^^ST AMINUT^NDIWILLTRYTOFINDTHEM» 



L,2 



„40 'U: RIGHT 



/ 
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41 o [outside interruption here] ^ 

^ Figure 7-la Dialogue Annotated for Requests - Part a 
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{^Ol it) 42 .O^ARE YOU STlLL THERE?^ . " 

•. ■ . ' . . • . ' ' ° & 

CQ.yi ^44 0: OK YES I HAVE FOUND THE FILES YOU ARE CONCERNED ABOUT^N THAT 

45 FIRST ONE I ASSUME THE ABDLLLL VtfAS AN ERROR RIGHT?\ 

• (M-h) 46 .H4JRIGH1)^HAT SHOULD HAVE BEEN [file namei].DAT;l AND 2. ' 



47 0: OK THEY ARE HERE 



48 ^ [outside interruption here] 
((XS')^9 O^ARE YOU THERE?^ 



{GlJ^-^ 50 «RIGHT» 



51 0: OK I HAVE FOUND THE FILES YOU WANT 

52 [outside interruption here] • ■ 

col) 53 0: OK I HAVE FOUND THE FILES YOU WANt|?WILL RETRIEVE THOSE FOR 

54 YGU ALSOJOK . ' ^ . 
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55 ,U: GREAT. 



Figure 7-lb - Dialogue Annotated for Requests - Part b 
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«. EXPRESSION OF COMPREHENSION 



In most dialogues, each participant wants the otKer to comprehend what he says. 
Since each person knows this, the participants often say things that serve to inform the 
other that some part of what was just said was comprehended, or not understood, or 
partially understood Sometimes there ar^ parts of the dialogue that perform only this 
function, and sometimes these parts do this while conveying other content. These are 
all aspects of Expressions of Comprehension that we want OBiservers to,comment on. 



There are two compohents of this mechanism for expressing 
Comprehension: there is the Expression itself, and there rs the part of ttie dialogue 
being referred to. In many cases, this second part is the other person's pre>^ious turn. 
We want the Observer to comment on both of these parts/ 

We have found that these Expressions occur frequently in two kinds of situations: 
those with noisy channels and those where there is a high cost for misunderstanding. 
We e.xpect tc^ find them whenever their cost to the speaker is lower than the cost of the " 
expected reduction in misunderstanding. ' ^ 

Here are the instructions for the Observer, 



OBSERVER'S INSTRUCTIONS FOR EXPRESSION OP' COM PREHENSIoll 

. We are interested in the Expression by a speaker that he compre^iiends 
. previous^dialogue. There ^re various ways in which -this Expression may 
occur: ^ ' ^ , 
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Three Explicit Examples:- 
"OK, That's clear." 



"Sure." 



A: "Is that clear?" 
B:"Yeah." 

O 

, An Implicit Example; 

A: "...so that proves the'lemma." 
B: "Using that same method, I can prove the second 

one." . 

■ ' ^» 

Our idea of comprehension here covers both hearing what is said and 
understanding it. We are interested in the hearer's Expression of hisi view of 
his own comprehension — the confide/ice dimension rather than the 
correctness dimension. (Thus these instructions will not serve to identify 
cases in which the hearer misconstrues but does not know 'that he 
misconstrues.) ^ > . . 

People in dialogue indicate various degrees or scopes of comprehension of 
what the other party has said. Using the directions below, we want. you to 
identify places where someone is indicatirrg his ^tate of comprehension of what 
has gone before, and to note the degree and sco^e of that comprehension. 

Three kinds, of indication of Comprehension will be noted; Positive 
Comprehension, Noncomprehehsion, and Selective Comprehension ' 



Positiva Comprelwmion 

Scan the transcript for explicit and implicit expressions of positive 
comprehension. Mark each one with ahglVvhpackets , < and >, and in 
parentheses the letters PC (for "positive comprehension") and a gnique 
njumber, e.g., (PC5). - , 

To qualify as an expression- of positive comprehension, a region must 
either explicitly express comprehension or implicitly indicate this by specific 
dependence on what was said before. The lack of an Expression of 
noncomprehension is not sufficient evidence for implicit positive 



comprehension. Instep, an utterance must rely on some specific knowledge 
expressed previously by 'the other person. 

Some of these Expressions not refer specifically to any definite part 
of the dialogue. 

* . ' ' • • ' ■' 

Others will clearly refer to some particular previous part. For the latter, 
mark the part(s) with the segment markers described in the general 
instructions , e.g. (, [, ], ) and »). Mark each of the$e markers with the PC 
/ number, e.g. (PC5) which it is indicating the scope of. If the scope of 
indication is exactly the previous comment |)y the other speaker, then a slash 
with the identifying number may be used instead of the scope brackets to 
indicate this; e.g. (PC5)/ . 

Multiple indications: Often people will indicate comprehension in more thgn 
one way (according to our notation). For example, they will commonly give an 
explicit indication s6ch as "Got that." followed by an implicit indication such as 
use 6f the comprehended material. 

RULE: Annotate the earliest indication. The scope of the indication should 
be the largest contiguous scope .which is either all Primary or a) I 
Nonprimary according to the directions given below. 

Use this rule for all the comprehension categories^ 

The comprehension may be expressed to various degrees: 

PI. Indefinite degree: Expressions like "Keep going" arid "I guess so" 
indicate?^ that some comprehension has occurred. 

. P2. Satisfactory or Substantially Complete Comprehension: 
Expressions like "OK" and "Sufe" -sometimes indicate this more complete 
level. Direct repetition without significant change of meaning is also used 
to indicate this level of comprehension. (But sometimes it indicates, 
selective cofiiprehension instead) Expression may include, approval pr 
. agreement or consent. . 

Mark eaph Expression of Comprehension wi|h an indication of the.degrpe 
of Comprehension expressed. Use one of the P numbers above,- or assigii a 
new P number of your own, and supply a descriptive phrase which t^lls 
qualitatively, the degree .of comprehension that was expressed. If you use 
your own phrase, try to express its ranK relative to the phrases above, e.g., 
more than PI but less than P2 



Noncomprahanaion ' 

Next, scan the v transcript ^ for explicit or Implicit' indications - of 
nohcomprehension. ' Phrases like "Huh?" and ';Say that again" may indicate 
noncomprehension. Mark each such indication 'with NC (for 
"nDncomprehen$ion"}^and a unique number, as above, and indicate the scope of 
thg, noncomprehended region, if known, and its partialness, if it is clearly 
-.partially noncomprehended Mark them in a manner directly analogous to the 
/ above directions for positive comprehension, . « 

Expressing a doubt about one's own comprehension is a variety of 
expressing noncomprehension. Even when in fact all was well comprehended, 
an Expression of doubt should.be marked as indicating noncomprehension. 

' • ■• : ^ - ■ 

Noncomprehension mgy be expressed to various degrees: . 

Nl. Indefinite degree of noncomprehension. 
N2. Substantially complete noncomprehension. 



Seleciiva Comprahamion 

People sometimes indicate what they have or have not comprehended. 
Where, they do so,' mark< the region with angle brackets and a unique SPC 
Vimber or SNC number (for "Selective Positive Comprehension" and ''Selective 
Noncomprehension" respectively) , as is done for PC numbers abdve. Use a 
new SPC or SNC number for each selected item, , To qualify as being selective 
comprehension, there must be some indication that there are differences in 
how well the various candidate^ for comprehen'sion have been received, ' 

Indicate the region for which the comprehension or noncomprehension is 
being indicated; as above, . % 

, . . - ' . i - 

If you identify selective positive comprehension within a region which you 
also mark as having noncomprehension, this will be interpreted as 
noncomprehension 6f all except the selected, item{s). Similarty, |ou« can show 
compreh(9nsion of all except selected items With a PC and an si\|C, for each: 

particular thing identified selectively as comprehended. 

•■ ', ■ if. " - ■ . ■■ 

Use the phrase marks, (such as PI and N2} from the Positive 
Comprehension and Noncomprehension directions to indicate categories of SPC 
and SNC. 




Th0 Primary/Nonprimary Dhtinction 

For each indication of comprehension that you Identified/decide whether it 
^ Is used by the hearer primarily to Indicate his comprehension/ or 
(alternatively) .whether indication of comprehension was performed along with 
V some other primary function. Mark the segment Identifying number with ++Jf 
J indication of comprehension is the primary function of the segment, and 
. / with - If it is not. So for example "huh?" might be marked "as (NC5)++/ 
showmig that Jt was Identified as an Indication of noncomprehension, and that 
the primary function of the segment was to indicate the speaker's lack of 
comprehension. 

Use of the material comprehended, such as answering a question, is a 
. common nonprimary indicatJon of comprehension. Simple Expression of 
\ ^ comprehension with approval or agreement or consent should be treated as a 
primarjpmdication of comprehension. 

Teopla sometimes indicate their state of Comprehension by telling ; the 
otheV persoi^ what to do about that state. Expres^ns like "Keep going- and 
"Say that again?" are used In this way. speaker's indication of his state of 
comprehension in any of these ways should be marked with ++. 7 

AN of these indteltors of comprehension deal with' a person -expressing 
- something about his -Siwn state. Do not use this notation for cases in which 
one person is indicating , that the other one is confused, has failed to 
comprehend, has comprehended well, or something similar. 

Notation summary: 

. Identify place where comprehension was expressed 

(■ > (i L ip )/-) " identify place which Expression refers to . 

segment expressing Positive Comprehension * 
NC segment expressing Noncomprehension ^ / 

SPC segment expressing Selective* Positive Comprehension 

SNC segment expressing Selective Noncomprehensida. ^ 

' expression of comprehension is primary function 

- >| • expression of comprehension Is not primary 7url£fion 
I (abbreviation) expresses comprehension of the prfevious 

' . comment by the other speaker 

Figure 8-1 shows an example Of applying these Instructions to a sample transcript. 



^7' • 



26 ,U:OK[operator's.name].(CAN YOU.PEOPLE GOINTOANY,DIRECTORY AND DELETE FILES 
. 27 NOTICE^THAT THEY Af?E NOT BEING USEdJ [user's name] • • 



28 'b(l AM REALLY NOT SURE>HOWEVER I KNOW THAT THAT WOOLD NOT BE 

29 RESP9NSIBILIT^<^E WOULD TAKE UF>ON OURSELVES TO JUDGE WHETHER OR. 
: 30 NOT YOU WANT YOUR FfL^S OR NOtl v\ . ■ 



31 [outside interruption here] -j f \ 

32 0: [user's name], ARE YOU THERE? ' j- 

33 LINK FROM [user's name], JOB 21^ TTY 22 



' 34 U^SORRY [operator's name] \ 

' 35 WE GOT DISCONI\IECTED)tCAN YOU RECOVER THOSE^F lES FOR ME.. 

_ 36 ,^S FAR AS I KNOW TflEY WERE IN THE DIRECTORY ON THE 16TH..,THE NAMES 

37 ARE v..[file namel].DATflABDLLLL ' CHECK THAT ...[file naty^].DATJl.2 / . *' 

' 38 [file name2].F4jl...}AND [file name3],F4ii„...[user'sn^ / '\ " - 

39 0<(OI^OLD ON JUST A MINUTE AND I WILL TRY TO FIND THEM; 

• . '^.i . ■> • ■ \ <w 

Figure 8-1 - Dialogue Annotated for' Expression of Comprehension 
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9. SIMILAR EXPRESSIONS /- mo KINDS 



need means for judging how well the 



OVERVIEW 

In modeling communication we particularly 
models we build perform comprehension or \nderstanding. Clues about what th^--^ 
understanding of dialogue depends |n are alsq vital. Developing .data on which those - 
■ judgments' can be made is an essential st?p toward 'making evaluations of coi#rehension 
operational. 

The Siniilar Ex^ssions category is intended to provide » a basis for knowing 

whether a modeJ ij^xtracting too much or. too little or the wrong things from particular 

^ items in a dialogue transcript. It is also intended to providecvery selective information 

about the dependencies- of interpretation that , arise between parts of an ongoing 
dialdguel • \ . 

We are |xploring this with two similar, but independent, experiments. Ths fit^st of 

.■■■»/ . _ ■ . ■ 

these calls, for the Observer to generate alternative expressions whici? he feels would 
accomplish the same function as existing expressions from a real dialogue. . In this case, 
he is given the full cont^t. of the surrounding dialogue.' The second approach is 
.significantly more complicated, involving the generation of these alternate expressions, 
but this time without the surrounding dialogue. The Observer is then given the 
surrounding dialogue and asked to separate the proposed alternatives«into those which 
will .work in the dialogue and those which will not. 
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, The details of these two experiments will be spelled out separately l^elow. 
However^ the: N ih each is the common, one of segmenting the given dialogue -into 

"units." This will be detailed first, with the understanding»that it is to be taken as the 



initial step in- each of the processes described subsequently. 



I tmructtons to Unit Divider . ' 

■ ^ You wilt be given a dialogue. Divide each turn into "units," where a unit is 
either a simple sentence, something ^hat functions like a simple sentence,, or 
the whole turn. Qo not feeJ bound to observe the punctuation you find ih the 
transcript it is likely to be inaccurate. The unit boundaries should reflect a 
single, coherent communication having approximately the "completeness" of a 
simple English sentence. In no case, however, is a .unit to be l^ger than a 
whole turn. Assign a unique number to each unit. | 



Similar Rxprossions In Contaxt 

- „We <vould like to compare different ways of saying something, i.e.. Ways 'that are 
rea^sonably construed as beirrg equivalent in communicatiofi effect. We will describe 
such expressions as' Similar and intend the term to have a specific techniqa! meaning.* 
^y comparing the ejects on the model of a dialogue item with the corresponding effects 
of the ^Similar Expressions, we can see whether the mtSdels are creating unnecessary 
differeo€e|OP-inissing important common effects. • 




OnSliRm'S INSTRUCTIONS FOR SiMlLAR EXP^RIiSSIONS IN 




We are interested in identifying different expressions that would se^rn to 
have the same effect ih a specific dialogue. » 

You will receive a transcript of the dialogue that has been divided into 
"units" (according to the directions, above, for Unit Dividers). For each unit, 
compose one or more Similar Expressions of each unit, consistent with the 
following constraint: ^ 
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If this new expression had occurred instead of the original unit, it < 
I ^i"ve been acceptable to the speaker for the purposes he had 

'"''3d- You should base your judgmenr only on the dialogue which „ 
^ precedes the . utterance in question, n&t on any dialogue which • 

follows it. . ■ „ •i^ 

/ . ■ " • ^ ■■• ■ ' ' "■'''•.*'■ 

. We encourage you to use different" words and styles of expression than 
appear in the given units. T^ese Similar Expressions constitute the results 
from the Observer, for this experiment. 

EXAMPLE OF SIMILAR EXPRESSION IN CONTEXT ' 

a 

STANDARD:. V • . / . * 

, Can you recover those files for, me? . ' , J 3 

SIMILAR EXPRESSIONS IN CONTEXT: , ^ 

• there any way to get my fil^s back? , ' " 

Please restore the$e files if possible, - . r 




*Two expressions are Similar if, in your opinion, the speaKer Would be 
willing to use the second at the point where he actually used thd first. 

We attempt to express this "definition-" a little more "clearly with tlio ' 
following hypothetical situation: Imagine a dialogue between ? speaker ahd "a 
hearer ^who aid not share a common language, but who had translators 
available. Imaging further >that in order to insure accuracy of communication, , 
the' speaker speaks to Translator 1 (only) who translates what he hears and 
communicates this tr'anslation to a second translator who cannot hear the 
original utterance. The second translator then retranslates the utterance 
„fronri Translator 1 back into the first language for the benefit of the speaker. 
Thus, the speaker is afforded the opportunity to hear his own words 
repeated back to him, after undergoing two passes of translation. At this 
point he has the option of accepting the echoed utterance as adequate, or 
rephrasing it and trying again. - 

Within the framework of tlnis hypothetical' situation, two expressioi?^!^ 
to be considered Similar if, after saying the first, -and subsequently hearing^ 
the second echoed through the second translator, the original speaker would 
be willing to permit the differences to g6 unchallenged' - that is, he accepts 
the different version as sufficiently adequate for his original purposes that it 
does not need revision. • 
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Similar ExprcSs%iom Out Of Context , , ^ 

. There are many different sources of informatidn that the InteYpretation of dialogue 
lext 

typically -depends , on. They include ' ^ 



. Situational lnformation- including the circumstances of the. dialogue, 
the identities and cultural roles of the participants,* 

Dialect arjd Jargon Information- The local conventions of language 
usage that were in effect at the beginning of the transcript 

•• - - » c 

o . • 

Context 'Information- The part of the transcript that precejdes the 
element being interprQted. 



interpretation, depends 6n each of these in the sense that for a given item 
^(serrlence, word, or other> variation in each 'of these related information sources xan 
produce variation in the interpretation of that item in dialogue. (Notice that the term 
"Context** is being used in a specific, narrbw and examinable way. It is not a catchall 
term for *'6ther effects.") ^ 



Because we are dealing explicitly with the effects' of each item in* Iheotranscript, 
Identifying effects df^^context is particularly importa^nt. These directions are designed 
to identify these effects and discriminate th^ from the others. 

The basit scheme for identifying context effects is sketched in Figure 9-1. 
Instructions - for each step appear below. The point of the process is -to generate 
expressions that are judged Similar in the out-of-context equivalence judgment, and 
then to judge the acceptability of these with the in-context equivalence judgment. 
THose th'at are unacceptable in this second stage must be so because of some eff&.ct of 
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ACTIONS 



Unit division 



' Scramble [>: 
and merge | 



Paraphrase 
, generation* 

Oift-of-context 
equivalence 
ludgemerit* 



Restordf-ioff 
to context 



Jn-coijtext . 
equivalence p 
lodgement^ ^ 



^erforrned by 
several people 




Acceptable 
in context • 



Figure 9-1 ^ Information Develc^pment for Similar Expressions Out of Context 
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context We ask the Observer in the -second judgment to describe factors that make 
suclt items unacceptable. We can then evaluate a model by examining whether a 
corresponding factor (difference of effect) arises when the proposed Similar Expression, 

rather than the original item, is read by the model at the appropriate moment 

' ' ■ ■ . <i * , ■ - 

. ' • • ■ . 

ousmvKws fmmvcTwm for shtiL/iR HXPKKssioNs our of conthxt 

_ We are interested in the effects of the preceding dialogue on the 
"•interpretation of a turn in that dialogue. In order to study these effects, we, / 
have devised a multistep procedure which makes use of your observations. It' 
starts witif , a dialogue transcript, isolates* pieces of some turns from their 
surrounding dialogue, identifies alternate. Similar Expressions, not -knowing the 
.context in which the unit originally occurred,' and finally implants these' 
. proposed alternates in the original dialogue at the point where the original 
piece had been, for judgment on how well or poorly these interpretations Ifit 

The Similar' Expressions are prepared Qut of context (i.e., without knowing 
what preceded or followed the original unit). The ones that turn out later to 
fit back info the context are use'd to develop methods of interpreting concepts 
that can be expressed in more than one way/ The Similar Expressions that 
are later judged not to fit are used to identify kinds of differences between 
various ways of saying things, leading to development of , methods for being 
responsive tq those differences. Both the Similar Expressions that fit into the 
context and those that faij are valuable. ' - 

(Several Observers will perform the unit division task on different 
dialogues. For each Observer in the next stage, we will select some units 
from each of thesQ dialogues and combine, them into a set having no particular 
order or relation between the unita-as they are presented.) 

SECTION 1: I nMrnr^ioHR to GoMarntor of Simitar Expres^^^ 

We are interested in different ways of saying something which have the 
same potential effect. We ^antf you to produce alternatives to some'' 
particular units that have occurred in actual dialogue. 

^You will receive a set of units. For each of these, we want you to write 
one 0/ more new units (Similar Expressions) ttiat are Similar to the originaf^ 
unit in the following wayr In your judgment, the new expressions would be ' 
regarded by the speaker as haying about the same ef'fect as his original unit in 
some common, ordinary circumstance. 



so. 
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• We encourage you- to use different words and styles of expression than 
appear m the original units. Only the effect which you thinK the speaker 
would expect needs to remain the same. " 

Generating these S'imilar Expressions might be easier if you proceed in the 
following steps: ' 

1. .Read the unit. . 

2. Imagine a situation in which the unit could occur. ' 

3. Imagine the effect intended by the speaker, 

- • 4. Invent a different unit which would have the same effect In the 
situation which you imagined. Wrtte this new unit down. 

5. Repeat steps 2 to 4 for as many different situations as come 
, readily to mind. (Note: "different" is to be interpreted as 

meaning situations which would suggest a different set of 

admissible Sinfiilar Expressions.) 



You should try to imagine ordinary situations rather than outlandish ones, 
since the expressions which you create will be judged by other Observers on 
the basis of whether they would have the same effect in some ordinary 
circumstances. • 

'\, '^^'■^®'<3'"P'e» in attempting to generate Similar Expressions for the unit: 
yes, the^ following are two common contexts and one we regard as 
outlandish* each followed by a set of Similar Expressions appropriate to that 
context: < 

I 

-1. ("Is Ac your boss?") He sure is./That he is./He's the one. 

• \- . 

2. ("Do you want to go to the movie?") You bet./Any time you're 
regdy./fine. 

3. . ("Vyhat's a thre^ letter word meaning°affih(iative?") Oui. 



- (For each unit, we will assemble a list of all Similar Expressions generated' 
m all contexis.by all Observers. The. next step Observers will be given 
several sets of these expressions, each with the original unit-that inspired that 
set. He still will not know the context in which the original unit occurred.) 



SECTION 2: fmtructiom to jad^e of Similar Expressions (out of context). 

We are infe^sted in whether various ways of saying sometliing^are Hkely 
to have the samp effect/ ° 

*You will receive a collection of units, divided into groups. One in each 
group will be called the" Standard Unit, and the others will be called the 
Comparison Units. Vou are to judge each Comparison Unit according to the 
following criterion: In your judgment, would the effect of the Comparison Unit 
• be acceptable to the speaker of the Standard Unit in some ordinary 
circumstances? . ^ 

Mark each Comparison Unit with one of the symbols: V\ "-" or "*" for 
Acceptable, Unacceptable, or Unclear. ' \ , 

(The Observer for the final step will be given one of the original 
transcripts with the units numbered. For each numbered unit, he will also be 
provided with a set of those Similar Expressions for that unit which received a 
V' in the prior step's grading.) 



SECTION 3: Instructions to judge of Similar Expressions (in context). 

oWe are interested in how the effect of what is said is related to the 
context in which it is said. We want you to make "some judgments about 
whether or not certain Similar Expressions for a unit can actually take the 
place of that unit in context, and, if not, why riot. 

Materials: 

You will receive a transcript of an actual dialogue, asr it occurred, divided 
into numbered units. For each numbered unit, you will also receive a set of 
possible alternative expressions for it. 

Commentary: 

For each one of the Similar Expressions given to you, make two kinds of 
annotation: 

1. If this expression had occurred instead of the original, would it have 
been acceptable to the speaker of the original? Mark it with eitJier V, or 

for Yes, No, or Unclear. You should project the acceptability of the turn 
based only, on the part of the .dialogue tliat precedes the unit, not an anything 
jA/hich follows. 
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2. If you answer then describe very briefly one important difference 
between the original and the alleged Similar Expression, from the point of view 
of the speaker. If the expression is incomprehensible in this context, mark it 
With X", meaning Incomprehensible (even if it might be meaningful in other 
contexts). 

These annotations constitute the results of the second experiment. \ 
« EXAMPLE OF SIMILAR EXPRESSIONS GENERATED OLIT OF CONTEXT 

STANDARD UNITr ^ 
As far as I know they were in the directory oh the 16th. 



RESULTS FROM SECTION 1: 

X. J %mk f*^0y^ kttt 

hsiU f^'tkt fin . 

•fie Itii. fVy. 

RESULTS FROM SECTION 1. . 

I. — 

CONTEXT PRECEDING STANDARD UNIT: 
• Can you recover those files for me? 

RESULTS FROM SECTION 3: 
f. 

I. f 
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10. TOPIC STRUCTURE 



The participants In a conversation often continue to talk about a particular set cJf 
concepts for several turns. This related set of ideas constitutes s Topic of discussion. 
• Topics may proceed for a long time, as in aMohg telephone discussion of vacation plans, 
or may proceed for only^wo short turns, as in a simple question-about the time its the 
Answer. Topics may be general or specific, and the participants may be discussing 
more than one Topic at a time. Topic is a unit based on the content being discussed 
rather than the forms being used. Some parts of the discussion may be part of ho 
cJearly distinct Topic - for exampl|, the negotiation of what to discuss next. The 
existence and nature of Units of discourse largeV than a 'single sentence is one of. the 
important issues being currently pursued by a number of researchers. The Topic 
annotation described here may shed light on the currently hypothesized supersentential 
structures, such as "frames'' (Minsky, i97A), "scripts" (Scharik & Abelson, 1975), and 
"story grammars'V(Rumelhart, 1975): 

Here is the current set of instructions for annotating Topics in dialogue: 



OBSKRVKn^S INSTRUCTIONS im CODING TOPICS 

In most:>d®&gti^s there are one or more Topics which are being xWscussed. / 
^ A Topic is a subject discussed in one or more turns of the dialogue. 

• . ■ ..'(■ . 
We want you to go through the dialogue, marking those segments in Vhich 
a Topic fs being discussed. Annotate t^^e^eginnipg and ends of these 
segments with the Segment .Markers, described above. Note that segments 
' may contain other embedded segments. 
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Do this for each person, separately In the dialogue. , Also, draw a line from 
the initial niarKer for each Topic to the margin (left for speaker. A; right for 
speaker Q) and write a brief descriptive title ^for the Topic. Use this same 
'' titi* whenever the same" Topic reappears, in either speaker's utterances. 
^lotice that not every part of the transcript has '^o be part of a Topic. •» 

When yq^ have finished annotating the Topics of the^ialogue, Itst any 
Topics still open at the end of4he transcript. Also list any Topics that were 
- already open when the part of the dialogue you have annotated started. To 
check these, make sure that every Topic marked as opened is either marked as 
being closed or listed as still open. Do the same for every annotation of Topic 
closing. 

Figurfe 10-1 is an example of ^n annotation resulting from the applteation 
of these instructions to a piece of dialbgue. 



\ ■ 
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53' 0: OK I HAVE FOUND, THE FILES YOU WANT I WIlL RETRIEVE THOSE FOR - . • 



54 . YOU ALSO, OK^ 



55 U: GREAtJ^AS THAT [name3] YOU WERE TALKING ABOUT? 



. — Pt^U ai iU 

"56 o7no, it is [name4], HOWEVER WHEN YOU "SEND MESSAGE acXtc 



57 SEND TO [name2] ' 

58 U: RIGHT...IS iname5] STILL AR[outside interruption here] 

59 IS '[name5] STILL AROUND. ., . ' . 

60 0: NO HE ISN'T. HE'S BEEN GONE FOR ABOUT THREE WEEKS NOW. 



PeooU J-^ OISItHANKS for your help , [operator's name] HAVE A GOOD DAY [user's name] OUt! 

^f-^'^ r— — — — — — —9i^uJiL 

62s 0: Oi<|YdU DO THE SAME WILL DO THOSE FILES RIGHT AWAY. BYE t'oMAuW 



63 U: THANKS AGAIW 



64 BREAK 



Figure 10-1 - Dialogue Annotated for Topic 
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SHARED TOPIC POST-PROCESStm ' • 

In our early directions, we attempted to have the Observer annotate the topic 
shared by the two participants. This led us into considerable difficulty, since the topics 
addressed by the individual participants were frequently "out of step" with each other 
in nontrivial ways. From our' attempt to'capture the notion, of one common topfc of a 
dialogue, we developed the ^concept of particular participant's, topic. Thus, the two 
participants may be talKing about different topics at the same time, as often occurs at 
topic boundaries, when one starts a new tbpic before ,the other finishes talKing about 
the Old one. This notion of a individuap topic is imbedded'in the insiructionk given 
above. 

' ■ .' f " . ■-- ^ ' _- 

However, we found that we could derive from these observations of an individual's 
topic an annotation of shared topic, (involving no additional subjective judgments),, 
following^he simple algorithngs given below. " V 



Post-ProcpMsitiff Procedure For Shared Topics , 

We >j^ill use the Observer's judgments of topic boundaries to determine the period 
^during which the two participants share a topic of conversation. ' 

Jhe genprah intention embedded in the algoritbm .is as follows: A participant is . 
sharing the topic°currently being discussed by his partner unles? the bbserver says, 
that he definitely is not. 



t Th6 detailed blow-by-blbw procedure is: 

: , * ■ • . # . \ ■ ,i. ' i 

1) To mark the beginning of a shared topic; 



a. Find a shared topic (T) (a topic discussed by both participants and given the 
same name by the Observer)/ 

b. Find the place that each definitely starts talking about T (the inner starts). 

c. t;alUhe;earl.ier start SE; call the later star^^^ 

•d For the person who spoke SL, .{PL), fin^d the Earliest point in the speech of 
either party at which: (SE had already ogcurred) and (PL had .not spoken 
anything which occurred prior to thp outer, start of topic T by PL), 
. This is the Topic Sharing Begin (TSB). 1 

' ■ i A ... ,1 

2) To mark the^end of a shared topic: 

. a. For each shared topic (T) marked as beginning, above, - , * 

b. Find the places where it definitely ends for'^ach participant -{the inner ends), 

c. Call the later end .EL; calj the earlier end £B / ' 

d. ^For the person who spoke EE, (PE), find the latest point at which (EL had not 
yet pccurre(C>Rd (PE had not spoken anything which occurred after the outer 
end of topic T by PE), * 

This is the Topic SJiar.ing En^: (TSE). 

3) ": If, after steps I and 2, it is disrcovered that the TSE prec^des'the TSB, then merely 
detete both marks. '. » " - 



4) The region of sharing of topic t.runs from the TSB to the Tsl:. 



The following examples are Intended tolHustr^te, in an abstract form, the desired 
intent of 'these directions. The five examples represent, two utterances eacli by A and 
B, alter:tiating, in the coarse of which a shared topic is-initiated. The segment mariners 
witWn the se^^juen^ A's and B's indicate their position within the dialogue dictated 
by the Topic Annotation directions. On the line immediately below this, the dashed line, 
indicates the region of the dialogue for whi^h the topic Js deemed to be shared. (Oh 



belief that start and end instructions are symmetric, only starts are showhO- 
1) lAAAAAAAAA (^BBBBBBBB kMKkfKMKlKkk BBBBBBBBBB ^ 



i- 



»2) fAA^AAAAAAA " BBBB(^BBBB AAAAAAAAAA BBBBBBBBBB 



3) AAAAAAAAAA BBBBBB^B AAAA^^AAAA BB(BBBBBBB 



4) AAAAAAAAAA BBB^BB(^B AAAAAAA'AAA BBBBBBBBBB 



— > 



5) AAAAAgW^AA BBBBBBBBBB AAAAAAAAAA (aBB^BBBB 



/ 



(n figure. 10-2, we exhibit an example of the. annotation that results from applying this 
procedure to the individual topic example given previously. 



0. 




^\ 'Hi 




Ox OK I HAVE FOUND THE FILES YOU WANT I WILL RETRIEVE THOSE FOR j " fijtttuyf^i^ 

' ■'■'] r* If " * 

YOU ALSO. OKJ ■ 



UlGRE^£VAS that [name3] YOU WERE TALKING ABO 
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59 



0:/N0, IT IS [name4], HOWEVER WHEN YOU SEND MESSAGE 
SEND TO [name2] 

U: RIGHT...IS [name5] STILL AR[outside interruption here] 
IS [name5] STILL AROUND. ' 




J 

& h airnet /y,'c 



60 0: NO HE ISN'T.- HE'S BEEN GONE FOR ABOUT THREE WEEKS NOW. 
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uT^ THANKS FOR YOUR HELP , [operator's name] HAVE A GQOq DAY [user's name] OUT. 

^ r . ^ ■ ° - - ' ■ : ' — — —tt<^U o^i^ 

0: OK)yOU DO THE SAME WILL DO THOSE FILES RIGHT AWAY. BYE . CiJi^otti^ 



63 U: THANKS AGAIN 



64 BREAK 
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Figure- 10-2 - Derived Shared-topic Annotations 
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11. INCREMENmi. AND PREREQUISITE INFORM /ITWN 

« . . . t ' 

006*0? the most difficult aspects of a dialogue to grasp (at least for our modeling 
. efforts) is what we loosely call the informational content. In order to somewhat confine 

our -Observers, we have specifically limited their annotations on the information 
' evidenced by the speakers, to that relevant to Jhe topics, previously annotated. Since 

we felt any choice of formal notation for informational content would sedously bias the 

ObsePvers, we-have opted .for simple English as the medium of expression for these 

olt)servations. 

Finally, we have asked the Observer to differentiate two kinds of information: that 
conveyed by the dialogue and that which was never expressed but must have been 
present for the dialogue to be understandable to the participants. 

. Although we are in full agreement concerning the importance of this particular 
category, we have ^|^pen unable to achieve ^pven a partial consensus on any proposed 
set of directions. We remain committed to the attempt to deal with this dimension but 
do not believe the current state of the directions merits being reported here. 
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12. COHUmTlON^ ACTfONS 

Every s& often in the course of a dialogue,' one of the participants becomes aware 
Of specific bit of misinformation which his partner seems to, be operating with. When 
the speaker sets abou1 to repair this unfortunate sjale of affairs, we say he is engaged 
m a Corre^ion Action, Whenever we find one of these actions, we would like to 
Identify*; * 



1) Where did the correction occur? 

2) What indicates tlial t+iis is a correction? 

3) What was being corrected? 

4) How do we know what is being corrected? 

5) How is the correct information indicated? 

6) How could the correction have been avoided? 

7) Did the recipient of the correction suggest it? 

8) Was the corrector, hibself, in error? 



OnSERVliR^S INSTRUCTIONS FOR CORRIiCTION /ICTIONS 

' We are interested in-how people, make corrections during the progress of 
a dialogue. We want to identify corrections that occur, gord eventually 
und©f stand how they are accomplished and what thei^ effects are. 

For the purposes of this set of observations, we want tO focus* only on 
particular kinds of corrections, ignoring all otiiers. The Correction Actions 
that we want you to identify and annotate must liave the following two 
properties: 



A. Retraction/Cancellation Property: : 

Something Tn the previous dialogue, which has been comprehended in some 
way by each participant, is retracted or cancelled by the further utterances of 
one of the participants. 
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Things retracted occur in a variety of forms. For example, the retraction may be 
effected by talking aboUt' retraction in some way, dr it • may occur as part of a 

substitution or revision. 

; ■ ■ ■ t ■ ■ ■ 

The thing corrected and the correction may occur in the same turn; in this case it is 
sufficient to find that^ in,the absence of the correction, the recipient would probably 
have comprehended the part of the coWnt being corrected in another way. 

The Correction Actibn may span severarturns by the person doing the correcting, 
Or it may be complete in a single tufnl 



B. Explicitness Property: % 

There must be some explicit indication that a Correction was intended (e.g.: 
-No, I meant ... "Sorry, make that ... ...). So repetitions, clarifications, 

explanatory elaborations and restatements are not usually Correction Actions. 

^ " . ^ . • ^ • . 

In our .experience. Corrections having these two properties are relatively rare. 

Many di^ogues do not contain any. Qn the other hand, .it is possible to find dialogues 
in which they are relatively frequent. 

The annotations arato be done m two stages, on separate copies of the trapscript. 
The first stage deals with the regions which do the correcting; the second, with the 
regions being. corrected. 
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ANNOTATIONS, FOR CORRECTION ACTIONS - first pass 
' A. CORRECTION REGION (CR) 

• > ■ . ^ ' . ' ■ ■ ■■ . ^ * . • • * ' - ■ . ■ ■ 

Before you can annotate any of the details of a Correction, you must first ascertain 
that a Correction has occurred. When you find a region of the Vialogue which satisfies 

the two requirements abo\m^(a Correction Region), bracket the region as you would a 

^ . ■ . • ' . * - . 

topic and assign the region a unique label: CR followed by a number (e,g. CR7). The 
Correction Region is to start with the first indication that a Correction is under way y 

(this would include a Torrectidn Request" by the recipient, see below) and end wl(en 

./•■._ 

the Correction is no longer under discussion, for example, 
. A: ... out window three, [correction on that! That's out window one!] 

^ -icii) 

Beyond this point,N^you should be Ware that ndt all of the requested annotations 
will apply to each Correction. Likewise, as usual, it may not always be obvious which 
classifications apply or where. In either case, you are simply to ignore the inapplicable 
instructions or those whose application is not obvious. ^ 

Underline and label each of the regions described below with their two-letter code 
(e.g.; CP, CC, CQ, ...) followed by the number assigned to the corresponding Correction 
Region. You might find that some of these ^regions overlap or are even identical with 
others—that's perfectly all right. 

^ ■ ■ J ' ' ■ ' ■ ^ • • 
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68 

a CORRECTION FLAG (CF) 

A "consequence of the Explicitness Property is that there is- almost always a part of 
the Correction Region (called the Correction Flag) in which the corrector signal that a 
Correction is taking place, (e.g! "Oops, what I meant was " , "Wrong-o!, the, d^ht 
answer is ...", ...) For each Correction Region you bracket, find its Correction Flag, 
example, / . 

A: ...out window three, [ correction on that ? That's out window one.") 




C. CORRECTION POINTER (CP) * " 

Within the Correction Region, the corrector will use certain words to indicate what 
part of the preceding, dialogue he wants to correct.. This region (the Corrtjctiot^^'*^ 
Fointer) will not represent any new or different information, for example, 

A:., out windpw three, fcorrection on that! That's out window one.]) 

<CP3) 

. .. .' ■ . • • . 

D. CORRECTION CONTENT (CO • 

In addition to locating the Error Region with the. Correction Pointer, the Correction^ 
Region will also inditate the nature of the substitution or revision. This novel 
information is the Correction Content, for exarfi'ple, ' 

Correction on that! That's out wir^dow one.J 
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E. CORRECTION REQUEST (CQ) . ' • ' 

Occasionally, the' recipient of an error will explicitly indicate his doubts that his 
partner indeed meanfwhat he clearly said, JEg.: "Did you mean or "ShouldnH" 
that be ... ...) If you see this happening, note this as a Correction Request. Note 

that this will tben be the beginning of a Correction Region, for example, 



--—Nj^: ... out window . thKee, 

B: £Are you sure you mean three? 



A: Correction on that! That's out window onej 
Figure 12-1 gives an example of first-pass Correction Action annotation. 

* r • ' 

ANNOTATIONS FOR COR^CTION ACTIONS ~ second pass ' • 

' ' - ■ ' ■ ■ ■ C \ 

On the second , copy of the' dialogue, write the bracj^ets and labels from the 
Correction Regions, as annotated in the first pass. Once this is done, proceed with 
annotating the regions described below. 

A. ERROR REGION (ER) . .. . . 

In most cases, the corrector will be making His CdrHrection as a modification of some: 
•prior region of the dialogue which contained the error. This part of the dialogue (the 
Error Region) may be In his own words, or his partner's. For each Correction Region, 
find the corresponding Ei^ror . Region, for example, , > ' ' 



A: ... out window three. (Correction on that! That's out window oftel 

} ■ r . 

A- 
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34 U: SORRY [operator's name] 

35 WE GOT DISCONNECTED..XAN YOU RECOVER THOSEi(l£S FOR 

36 AS FAR AS I KNOW THEY WERE IN THE DIRECTOR^N Thi 16TH...THE 

37 ARE ...[file namegDATjlABPLLLL f CHECK-THAT . ..[file name 13.D AT; 1 ,21 

38 [file name2].F4il....AND [file hame3].F4il....|user's name] / C C /f ^ ) 



39 . 0: OK HOLD ON JUST A MINUTE AND I WILL TRY TO FIND THEM ' 



40 U: RIGHT 



41 [butside interruption here] 



42 0: ARE YOU STILL THERE? 



43 U: RIGHT [operator's name] 



44 0: OK YES I HAVE FOUND TflE FILES YOU ARE CONCERNED ABOUTj ^N TtiAT 



45 FIRST ONE I ASSUME THE ABDLLLL WAS AN ERROR RIGHT? 



4tf U; RIGHT THAT SHOULD HAVE BEEN [jile namel].DAT;l AND 2.] 



47 0; OK THEY ARE HERE 



Figure 12-1 - First-pass Correction Action Annotat 



ions 



The error region should include not only the text which is actually/correctecl, but* 
also that which is used by theOorrection Pointer (see below) to locate ^he area to 
corrected. ' / 

Sometimes,, the corrector is not correcting preceding dialog|ue, but rather an 
erroneous concept which he believes the recipient holds. If you see this happening, 
describe this erroneous concept in the margin and treat it as the Error Region. 

B. CORRECT ALTERNATIVE (CA)' | 

Frequently it is easy to ?ee how the error which was corrected could have been 
avoided in the first place. If possible, select a region of the dialogue (usually the Error 
Region) and rewrite, it, using as nearly as you ran the.style of the speaker, so that, had 
the substitute been used, the net effect Would have been the same as the actual 
.utterance and subsequent CoVre^tion, for example, ^ 

^ - A: ... out window three . (Correction on that! That's out window one J 



C. CORRECTOR ERROR (CE) ' 



(C/93) 

Cot UifftoiA^' 



If you encounter a situaticfn where, in your judgment, the corrector makes a 
Correction which, if successful, will have no net effect, indicate this with a STAR (*) in 
the margin, alongside the Correction Region, for example,' 
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Turn off switches one, two and seVen. 
O.K., will do. 
(theck that^ switch two should also be ofQ 
Roger, switch two ofif. 



Figure 12-2 shows an exampte of second pass annotations for Correction Actions. 
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34 U: SORRY [operator's name] . . ' - 

35 , WE GOT DISCONNECTEO:...CAN YOU RECOVER THOSE RLES-. FOR. ME. 
AS FAR AS I KNOyV THEY WERE IN THE DIRECTORY ON THE- 16TH...THE NAMES 
ARE ...[fite namel3.DAT;lABDLLLL [^CHECK THAT ...[file name^lJDATj 1,2]. 

's name] " 

39 0: OK HOLD ON JUST A MINUTE AND. I WILL TRY TO FIND THEmT ' -o. 



36 
37 
38 



[file name >].F4;1....AND [file nal 



i|ne3].F4;k,..[user's 
^ (€TIX), 



CCitd) 



40 U: RIGHT 



41 [outside interrupti(yn here] 



42 0: ARE YOU STILL -THERE? . 



43 , U: RIGHT [operator's name] 



44 0: OK YES I HAVE FOUND THE FILES YOU ARE CONCERNED ABOUTjONm 



45 FIRST ONE I ASSUME THE ABDLLLL WAS AN ERROR RIGHT? 

46 .U: RIGHT THAT SHOULD HAVE BEEN [file namel].DAT;l AND zl 



47 0: OK THEY ARE HERE 



Figure 12-2 - Second-pass Correction Action Annotations 
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13.' SUMMARY AND PLANS 



The report above has described the very subsj;antiarprogress that has been made 
Off Identifying suitable dialogue phenomena for this w;ork. .The observation methods are 

all in a state of incomplete development, which will' necessarily pe/sist for some time. 

* ' ■ ■• ' ' ■ ' J, ' * 

Since they are not in a finished state, it is not timety to try to make^ them;, completely 

adequate for widespread use or formally assess their reliability.' 

♦* ' . • , ' . ^ 

The substantive content of,/ the observational categories must be fitted to xne 
modeling process in ord^r to havl^b a smoothly bperatihg methadolggy. This fitting has„ 
not yet been done- Until it is, the basic definitions of the categories will rtot be stable. 
Therefore formal, val'^^n worl^ or extensive documentatfon work on the^'observatio.nal 
methods is not appropriate at this time. The impact of later modeling activity woufd 
effectively cancel any imir^ediate work of either kind. ^ 

On. the other hand, there is value' in exposing the methodology and observational 
categories to technical\nd personal points of view not represented among the authors. 

We plan to construct process models for one or tWo very short dialogues 
immediately. The purpose of these models \lill be to sketch the form of the processes 
needed and detertnine major relationships between parts. This stage will* necessarily 
Jnvolve a subset of the Observers' categories, relatively rough representation of some 
kinds of knowledge, and short dialogues. ^ 



As our modeling skill develops, we expect to deal wi|h more phenomena per models 
"* -. 
'(with longer* dialogues, and with higher fidelity to the actual communication events. 

. » ■ ■ ■ /■ . 

- • ■ ; ■■■ ^ .• • / . 

V f^ast experience on related problemjs indicates that/such models often converge 

rapidly an some relatively effective processes, each episode of accounting for new data 

■ " . ,' ' . • '/•*''' * ■ ' 

ta(sing substantially less work than the previous one. When this . convergence takes 
place, it is indicative of success. Also, ability to model rapidly makes it possible to build 
successively more ambitipus models. 

■ . > ; . . * ■ , ■ _ . ■ ■ . 

. 4 .•• . . / • • . ■ ■ ■ ' . . • 

'"<'■,'. ' ' t *" • , ^■■ ^ ■ ■ - ' " 

Different parts Of these models will mature at differetit rates. It is to be expected 
that some processes will be 'ready for transfer into working man-machine systems 
relatively soon, and others much later We plan to identify and report on those that 
appear to be ready for transfer on a continuing* basis. 




APPENDIX 

FULL TEXTS OF TIIE DIALOGVE OF TflE EXAMPLE 



.0 \ 



The examples used in the report are pot character-by-character reproductions'of 
the original' typed dialogue. Some differences have been introduced for various 
reasons, includiflt -|i»;iyacy,y^^^^ 0/ use, and' a desire to focus on certain phenomena. 
This^ection ma^es iV possible to. identify 'some of the differences by showing a text 
which more strongly resemble!^ the source text^ but with privacy preserved and genuine 
irreleVancie^ .dejeted. (The text which resembles the source is shown, first, then the 
cleaned-up versron prepared for tl^e Observer.) ' ' 

We have chosen to use cleang,d-up data in order to limit the diversity of phenomena 
which need to be dealt with at once. Because, the differences Introduced in this way ' 

■ ^ . ^ * ' . ^ ■ ■ 

seemr distinct fcom the phenomena of central interest, is therefore" a reasonable tactic ^to 

■ . a . . . ■ ■■° ■ „■ ■ 

elimirtate them in the immediate future, without in any way ruling them out of the scope 

■ ■ ~ ' ' • ■ ' * . 

Of the general problem of understanding how communication works 

■'■.*■ . 
■ . ' " {)■ ' ■ • ' ' • 

The transcript below; represents communication between a computer operator and « 
user of the computer. Their terminaTs have, been linked together by use of the LINK 
t^nmsnd, whlich causes each character which appears at either termi/al to appear at 
.the other as well. The transcript differs from the Original only in the following ways: 

Material deleted is noted by {square brackets] . . i 

2. "0" indicates the operator's comments, and V tlie us^^^ / ' ^ 

3. Each turn fey a speakbr has been set off by a blank line and these labels. ^ 



Jhe spelling, grammar,, punctuation, etc. ' are those that originally 
occurred. . • " . ^ 

' ■ " i.* . • * ' ■ > , * 

t ' ■ \ ■ . ■ . . 

■■ • '• ■ ■ ■ 

' . '.'i ' ' ■ ■ ' ' """" "~" - ,- ■ 

f i: LINK FROM [user's ^n ' , « • 

^ :/ 9 -UHELLO^AREYOU THERE...GA . . . . • 

' ' ^ ' : ■ ■ ^ ' • " ■ ■ 

10 0: YES, GO AHEAD ° 

11 U: A COOPLE A QUESTIONS....ARE0THE FILES. THAT ARE ON -DISK ARCHIVED EVERY 

12 WEEK/ " • 

^ ■ ■ ■ ' • - - ' ^ ' . ' 

13 0: YES • ' • 

14 U:IXXX • ^ 

15 , IN ONE OF- MY DIR. I HAD SOME D/ FILES IN ONE DAY AND THE NEXT THEY WERE 
.16 GONE AND f HEY WERE NOT ARCHIVED/ , . ° ■. 

■ ; • , - ■ . / - . 

17 0: WELL, I AM REALLY NOT THAT FAMIL SURE OF HOW THE ARCHIEING OF 

18. THE FILES ARE DONE SO I WOULD SUGGEST THAT YOU SEND A MESS. TO 

19 [name2] IN REQ\QGARDS TO. THIS PROBLEM. THE ARCHIE\EVEING IS DONE 

,20 DURING THE SWING SHIFT SO IN ORDER THAT YOU DO DONT NOT GET 

21 MES MISLED BY SOMETHING I AM NOT CERTAIN ON I WOULD RATHER ' 
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22 . YOU DISCUSS THIS ^^4+fTratne2i^S OUT OF ijiS OFFICE AT THIS 

23 TIME BUT YOU COaO SEND HIM A MESS^NIXk^URE HE S WILL GIVE 

24 YOU ALL THE INFO YOU WOULD LiIe. [opdra 



r's na^e] 



25 U: OK-toperator's name]. CAN YOUPEOPLE GO IN TO ANY DIR AND^DEL V/ FILES WHEN YOU 

26 NOTICE THAT THEY ARE NOT BEING USED., [user's name] • 

27 0: I AM REALLY NOT SURE, HOWEVER I KNOW THAT THAT WWQULD NOT BE 

28 RESPONSeUTY WE WOULD TAKE UPON OURSELVES TO JUDGE WHETHER OR . 

29 Y NOT^OU WANT YOUR FILES OR NOT. 

30 [outside Interruption here] . i 

. - ^ ■ o 

.31 " 0: [user's name], ARE YOU THERE? ' ' . . " 

:V ■ ■■■■ ■■ 



32 



LINK FROM [user's name], JOB 21, TTY 22 



^, 33 U: SOER " ' %. 

34 RY [operator's name] 

WE C/ GOT DIC/SCONNECTED....CAN YOU RECOVER THOSE FILES FOR ME.. ' 
AS FAR AS I KNOW THEY E/WR// WERE IN THE DIR ON THE I6TH...THE NAMES . 
ARE ...[file name; ].DAT;1ABDLLLL////////A/ CHECK THAT ...[file name 1].DAT;1,2 
[file name2].F4;l....AND [file name3].F4jl [i/ser's name] 



35 
36 

r 

37 
38 



I 
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39 Os OK HOLD ON JUST A M!N. AND I WILL TRY TO FIND THEM 



40 U; RIGHT- 



a I 



41 [outside interruption here] 



42 0: ARE YOU STILL . THERE?' 



43 U: RIGHT [operator's narfie] . ^ 

■ • '% ' ' ■ . " 

44 0: OK YES I HAVE FOUND THE FILES YOU ARE CONECERNeS ABOUT, ON THAT 

45 FIRST ONE A ASSUME THE ABDLLLL .WAS AN ERROR RIGHT? 

. ■ • . ; ■ , . -i^ 

46 U: RIGHT THAT SHOULD HAVE BEEN [file bamei].DAT;l AND 2/ 



47 0: OK THEY ARE HERE 

48 [outside interruption here] 



49 0: ARE YOU THERE? 



50 BRIGHT 



/ ■ 
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51 0: OK I HAVE FOUND THER\R Y\Y THE FILES YOU WANT 

. . • / * ■ 

* • : . o 

52 ' [outsfde interri}|^,Jio^ 

o • . . . J 

. • • y 

53 0: OK II HAVE FOUND THE FILES YOU WANT I WILL RETRIEVE THOSE FOR 
.54 . YOUXXX ' ' ^ 

55 • YOU ALSGj'OK , 

. ■ • t ■ . ■ 



• 56 U:'t3RA/EAT..WAS THAT [nameS] YOU WERE TALKING AV/BOUT? 



57 0: m,^T 0\0 jS [nanrie4J .HOWEVER WHEN YOU SEND MESS 
,58 . SEND TO [n'ame2] „ ' • « 



59 U:=RIGHT.JS [name51 STILL AR[outsidernt^rruptipn here] i 

60 IS [naiVie5] STILL AROuijD. " ■ » ' 

61 0: NOHE ISN'T. HDS BEEN GONE FOR ABOUT THREE WEEKS NOW. 



» ■ • ^ ^ ^ 

62 U: OK, THANX F^R YOUR HEL^,, [operatpr's name] HAVE A GOOD DAY [user^s name] OUT. 

• • t»j . * . ' ■ ' , « 

I ! ■ ■ ' ' "■'■->►■• 

63 0: OK YOU DO THE SAME WILL DO THOSE FILES iit^^ BYE ' ' , . 

» • ■ ' .;■ ^ • ' I - ' ■ i ■ 



• 64 U: IHfm AGfflN "l 



65 BRA\AEAK ' . ^ . ' . 

. ■ .. ' - J 

The transcript below is the cleaned up version. 

10 LINK FROM [user's namej, JOB 21, TTY 3 ^ 

11 U: HELLO ARE YOU THERE . " ' . 

12 0: YES, GO AHEAD ■ . 

■ -/ ■ 

13 U: A COUPLE- A QUESTIONS....ARE THE FILES THAT" ARE ON DISK ARCHIVED EVERY 

14 WEEK? ^ ^ 'V 

15 0: Y^S . • " 

- . , ■. ^ . * 

I" ' ■ . _ « ' - ■ . 

16 U: IN ONE OF MY DIRECTORIES I HAD SOME' fFlES IN ONE DAY AND THE NEXT THEY WERE 

17 GONE AND THEY WERE^NOT ARCHIVED " 

18 0: WELL, I AM REALLY NOT THAT SURE OF HOW THE. AfecHiyEING OF • 

t r -■ 

19 . THEFILES ARE DONE SO I WOULD SUGGEST THAT YOU SEND A MESSAGE TO 

20 [name?] TN REGARDS TO THIS. PROBLEM. THE; ARCHIVEING IS DONE: 

21 ^ DURING THE SWING SHIFT ORDER THAT YOU DO DONT NOT 3ET 

22 \ Misled;' BY soivIething i am not certain on i would rather 

23 YOU DISCUSS THIS WITH [name2]. ,I^E IS OUT OF HIS OFFICE AT THIS - 

24 ' TIME BUT YOU COULD SEND HIM A MESSAGE AND I AM SURE HE. WILL GIVE 

25 . YOU ALL THE. INFORMATION YOU WOOLD LIKE, [ope'rator'l. name] 
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26 U: OK [operator's name]. CAN YOU PEOPLE GO INTO ANY DIRECTORr^ND DELETE FILES WHEN YOU 

27 NOTICE THAT THEY ARE NOT BEING USED., [user's name] 



28 0: 1 AM REALLY NOT SURE, HOWEVER I KNOW THAT THAT WOULD NOT BE 

29 ^ RESPONSIBILITY WE WOULD TAKE UPON OURSELVES TO JUDGE WHETHER OR 

30 • NOT YOU WAj^T YOUR FILES OR NOT. ' 

~' ■ < ■ ■ ■ ■■ • ". V 

31 [outside interruption here] ■> 

32 Oi [user's name], ARE YOU THERE? 



33 LINK FROM [user's name], JOB 21, TTY 22 

■ / .- ■ ■• ■■■■■ 



/ 



34 U: SORRY [operator's name] • 

WE GOT PISCONNECTED....CAN YOU RECOVER THOSE FILES FOR ME.. 



35 
36 
37 
*38 



AS FAR AS I KNOW 1|EY ,WERE IN THE DIRECTORY ON THE 16TH.,.THE NAMES 
ARE. ...[file na'melJDATilABDLLLL CHECK THAT ...[file namel].DAT;l,2 
[file name2].F4il....AND [file name3].F4il [user's name] <, 




39 lO'. OkIhOLD on JU&t a minute and I WILL TRY TO FIND THEM 



ERIC 



40 U/RIGHT 



I - . 



41 -IplrtsiaeTnfeirr^Uon here] 



X 
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42 0: ARE YOU STILL THERE? ^ 



43 U: RIGHT [operator's name] 



44/0: OK YES I HAVE FOUND THE FILES YOU ARE CO^ICERNED ABOUT, ON THAT 
45 FIRST ONE I ASSUIi|g'THE ABDLLLL WAS AN ERROR RIGHT? 
Ae U: RIGHT THJ^^^^Ul^^^^ 2. 



47 0: OK TH^Y ARE HERE 



A. 5 



\ ■ 5 



48 [outside ih^terruption here] 



49 0: ARE YOU THERE? 



50 U: RIGHT 



J' 



51 OK I HAVE FOUND THE FILES YOU WANT 

52 [outside interruption here] " v • ' " 

53 0: XDK I HAVE FOUND THE FILES YOU WANT I WILL RETRIEVE THOSE FOR 
-'54 >■ YOU ALSO, OK . '^i \ /•■ 



55 U: GREAT..WAS THAT [namea] YOU WERE TALKING ABOUTf 



/, ■■■ ■ :■ .■;/' 

, so , 
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56 0: NO, IT IS [name4], HOWEVER WHEN YOU SEND MESSAGE 

57 SEND TO [name2] 

58 U: RIGHT..;IS [name5] STILL AR[outside interruption here] 

59 IS [naroe^j STILL AROUND. 



60 0: I^O^HE ISN'T. H^S BEEN GONE FOR ABOUT THREE WEE<^ NOW. 



61- U: OK, THANKS FOR YOUR HELP , [operator's na?»iej HAVE A.GOOD DAY [user's name] OUT. 



62 0: OK YOl/DO THE SAME WILL DO THOSE FILES RIGHT AWAY. BYE 



63 U: THANKS AGAIN 



64 BREA 



J 
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ERIC . . 



; . . :• ; : - /. ■'■. ■ •.. - ■■■ 

Mann, W. C. Man-Machine ^ Communication. Process. Rese^irck 
^ (a proposal from) USC/lnformation Sciences Institute to 

ARPA HRRO, October 1974. » . ' . , ' 

Minsky, U A Framework for Represehting"Kno'wledge. M.I.T. Al Me^rto 
. - 306, 1974. ; ' 

Riumelhart, a E Notes on a Schema for Stories. In D. G. BobroW, & - 
> ' A, Ml Collins (Eds.), K<?preji<»mano« and Undarsytndi^g, 1975/ 

Schank, R.:C., i?r Abelson, R. P. Scripts, Plans, • and Knowledge, 
Submitted to Imcmational Joim Coft^^ on Artificial 
Intalligencc, 1975. \ ^ / ' ^ 



